A SILICON IMPLEMENTATION OF A NOVEL MODEL FOR RETINAL PROCESSING A Dissertation

A SILICON IMPLEMENTATION OF A NOVEL MODEL FOR RETINAL PROCESSING A Dissertation
A SILICON IMPLEMENTATION OF A NOVEL MODEL FOR RETINAL
PROCESSING
Kareem Amir Zaghloul
A Dissertation
in
Neuroscience
Presented to the Faculties of the University of Pennsylvania
in Partial Fulfillment of the Requirements for the
Degree of Doctor of Philosophy
2001
Dr. Kwabena Boahen
Dr. Michael Nusbaum
Supervisor of Dissertation
Graduate Group Chairman
COPYRIGHT
Kareem Amir Zaghloul
2001
For my parents
iii
.
iv
Acknowledgments
I would like to acknowledge and to thank all of the people who I have come to know
and who I have come to depend on for support and encouragement while embarking on this
incredible journey:
First and foremost, I would like to thank my advisor, Kwabena Boahen. I thank him
for his mentorship, for his encouragement, for his patience, and for his friendship. I thank
him for teaching me, for pushing me, for having confidence in me, and for supporting me.
He has taught me more during this time than I could have imagined, and for that I will
always be grateful.
I would like to thank Peter Sterling, who at times served as my co-advisor, but who also
served as my co-mentor. I thank him for his wisdom, for his advice, for his encouragement,
and for his faith in me.
I would like to thank Jonathan Demb, who I have worked with so closely over the past
several years. I thank him for his guidance, I thank him for all his help in my pursuit of
this degree, and I thank him for teaching me what good science is all about.
I would like to thank the other members of my committee: Larry Palmer, Leif Finkel,
and Jorge Santiago. I thank them for their constructive criticisms, for their help, and for
their support.
I would like to thank the members of my lab who have all, in one way or another, helped
me tremendously during this endeavor. Some have been there from the beginning, some are
new, but all have made this entire experience incredibly enjoyable.
v
Finally, and most importantly, I would like to thank my family and my friends who
supported me and stood by my during every step of this journey. Without their encouragement, without their faith, and without their love, I would not have found the strength to
continue. This thesis is as much for them as it is for me.
vi
Abstract
A SILICON IMPLEMENTATION OF A NOVEL MODEL FOR RETINAL
PROCESSING
Kareem Amir Zaghloul
Kwabena Boahen
This thesis describes our efforts to quantify some of the computations realized by the
mammalian retina in order to model this first stage of visual processing in silicon. The
retina, an outgrowth of the brain, is the most studied and best understood neural system. A study of its seemingly simple architecture reveals several layers of complexity that
underly its ability to convey visual information to higher cortical structures. The retina
efficiently encodes this information by using multiple representations of the visual scene,
each communicating a specific feature found within that scene.
Our strategy in developing a simplified model for retinal processing entails a multidisciplinary approach. We use scientific data gathering and analysis methods to gain a better
understanding of retinal processing. By recording the response behavior of mammalian
retina, we are able to represent retinal filtering with a simple model we can analyze to
determine how the retina changes its processing under different stimulus conditions. We
also use theoretical methods to predict how the retina processes visual information. This
approach, grounded in information theory, allows us to gain intuition as to why the retina
processes visual information in the manner it does. Finally, we use engineering methods to
design circuits that realize these retinal computations while considering some of the same
design constraints that face the mammalian retina. This approach not only confirms some
vii
of the intuitions we gain through the other two methods, but it begins to address more
fundamental issues related to how we can replicate neural function in artifical systems.
This thesis describes how we use these three approaches to produce a silicon implementation of a novel model for retinal processing. Our model, and the silicon implementation
of that model, produces four parallel representations of the visual scene that reproduce the
retina’s major output pathways and that incorporate fundamental retinal processing and
nonlinear adjustments of that processing, including luminance adaptation, contrast gain
control, and nonlinear spatial summation. Our results suggest that by carefully studying
the underlying biology of neural circuits, we can replicate some of the complex processing
realized by these circuits in silicon.
viii
Contents
1 Introduction
1
2 The Retina
8
2.1
2.2
Retinal Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
2.1.1
Cell Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
2.1.2
Outer Plexiform Layer Structure . . . . . . . . . . . . . . . . . . . .
13
2.1.3
Inner Plexiform Layer Structure . . . . . . . . . . . . . . . . . . . .
19
2.1.4
Structure of the Rod Pathway
. . . . . . . . . . . . . . . . . . . . .
21
Retinal Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22
2.2.1
Outer Plexiform Layer Function . . . . . . . . . . . . . . . . . . . .
22
2.2.2
Inner Plexiform Layer Function . . . . . . . . . . . . . . . . . . . . .
26
ix
2.3
Retinal Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
29
2.4
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
33
3 White Noise Analysis
34
3.1
White Noise Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
3.2
On-Off Differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
50
3.3
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
60
4 Information Theory
62
4.1
Optimal Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
63
4.2
Dynamic Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
75
4.3
Physiological Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
83
4.4
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
86
5 Central and Peripheral Adaptive Circuits
88
5.1
Local Contrast Gain Control . . . . . . . . . . . . . . . . . . . . . . . . . .
89
5.2
Peripheral Contrast Gain Control . . . . . . . . . . . . . . . . . . . . . . . .
104
5.3
Excitatory subunits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
115
x
5.4
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6 Neuromorphic Models
120
125
6.1
Outer Retina Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
128
6.2
On-Off Rectification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
137
6.3
Inner Retina Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
140
6.4
Current-Mode ON-OFF Temporal Filter . . . . . . . . . . . . . . . . . . . .
154
6.5
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
165
7 Chip Testing and Results
167
7.1
Chip Architecture
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
169
7.2
Outer Retina Testing and Results . . . . . . . . . . . . . . . . . . . . . . . .
176
7.3
Inner Retina Testing and Results . . . . . . . . . . . . . . . . . . . . . . . .
183
7.4
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
196
8 Conclusion
198
A Physiological Methods
204
xi
List of Figures
2.1
Different Layers in the Retina . . . . . . . . . . . . . . . . . . . . . . . . . .
12
2.2
The Flow of Visual Information in the Retina . . . . . . . . . . . . . . . . .
14
2.3
Rod Ribbon Synapse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
2.4
Structure and Function of Major Ganglion Cell Types . . . . . . . . . . . .
27
2.5
Quantitative Flow of Visual Information . . . . . . . . . . . . . . . . . . . .
32
3.1
Linear-Nonlinear Model for Retinal Processing . . . . . . . . . . . . . . . .
37
3.2
White Noise Response and Impulse Response . . . . . . . . . . . . . . . . .
40
3.3
System Linear Predictions . . . . . . . . . . . . . . . . . . . . . . . . . . . .
42
3.4
Mapping Static Nonlinearities . . . . . . . . . . . . . . . . . . . . . . . . . .
44
3.5
Spike Static Nonlinearity . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
xii
3.6
Predicting the White Noise Response . . . . . . . . . . . . . . . . . . . . . .
47
3.7
Ganglion Cell Responses to Light Flashes . . . . . . . . . . . . . . . . . . .
49
3.8
Normalized Impulse Responses . . . . . . . . . . . . . . . . . . . . . . . . .
51
3.9
Impulse Response Timing . . . . . . . . . . . . . . . . . . . . . . . . . . . .
53
3.10 Normalized Static Nonlinearities . . . . . . . . . . . . . . . . . . . . . . . .
54
3.11 Static Nonlinearity Index . . . . . . . . . . . . . . . . . . . . . . . . . . . .
56
3.12 Normalized Vm and Sp Flash Responses . . . . . . . . . . . . . . . . . . . .
57
3.13 ON and OFF Ganglion Cell Step Responses . . . . . . . . . . . . . . . . . .
59
4.1
Optimal Retinal Filter Design . . . . . . . . . . . . . . . . . . . . . . . . . .
65
4.2
Optimal Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
68
4.3
Power Spectrum for Natural Scenes as a Function of Velocity Probability
Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
70
4.4
Optimal Filtering in Two Dimensions . . . . . . . . . . . . . . . . . . . . .
72
4.5
Contrast Sensitivity and Outer Retina Filtering . . . . . . . . . . . . . . . .
74
4.6
Dynamic Filtering in One Dimension . . . . . . . . . . . . . . . . . . . . . .
77
4.7
Inner Retina Optimal Filtering in Two Dimensions . . . . . . . . . . . . . .
79
xiii
4.8
Retinal Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
84
4.9
Intracellular Responses to Different Velocities . . . . . . . . . . . . . . . . .
85
5.1
Recording ganglion cell responses to low and high contrast white noise . . .
91
5.2
Changes in membrane and spike impulse response and static nonlinearity
with modulation depth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
94
5.3
Scaling the static nonlinearities to explore differences in impulse response .
96
5.4
Root mean squared responses to high and low contrast stimulus conditions
97
5.5
Computing linear kernels and static nonlinearities for two second periods of
every epoch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
100
5.6
Changes in gain, timing, DC offset, and spike rate across time . . . . . . . .
103
5.7
Recording ganglion cell responses with and without peripheral stimulation .
106
5.8
Unscaled changes in membrane and spike impulse response and static nonlinearity with peripheral stimulation . . . . . . . . . . . . . . . . . . . . . .
108
Scaled ganglion cell responses with and without peripheral stimulation . . .
109
5.10 Changes in gain, timing, DC offset, and spike rate across time . . . . . . . .
113
5.9
5.11 Unscaled changes in membrane and spike impulse response and static nonlinearity with central drifting grating . . . . . . . . . . . . . . . . . . . . . .
xiv
116
5.12 Scaled ganglion cell responses with and without a central drifting grating .
119
5.13 Comparing gain and timing changes across experimental conditions . . . . .
122
5.14 Pharmacological manipulations . . . . . . . . . . . . . . . . . . . . . . . . .
124
6.1
Morphing Synapses to Silicon . . . . . . . . . . . . . . . . . . . . . . . . . .
127
6.2
Outer Retina Model and Neural Microcircuitry . . . . . . . . . . . . . . . .
129
6.3
Building the Outer Retina Circuit . . . . . . . . . . . . . . . . . . . . . . .
134
6.4
Outer Retina Circuitry and Coupling . . . . . . . . . . . . . . . . . . . . . .
136
6.5
Bipolar Cell Rectification . . . . . . . . . . . . . . . . . . . . . . . . . . . .
138
6.6
Inner Retina Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
143
6.7
Effect of Contrast on System Loop Gain . . . . . . . . . . . . . . . . . . . .
146
6.8
Change in Loop Gain with Contrast and Input Frequency . . . . . . . . . .
149
6.9
Inner Retina Model Simulation . . . . . . . . . . . . . . . . . . . . . . . . .
152
6.10 Inner Retina Synaptic Interactions and Subcircuits . . . . . . . . . . . . . .
155
6.11 Inner Retina Subcircuits . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
160
6.12 Complete Inner Retina Circuit . . . . . . . . . . . . . . . . . . . . . . . . .
162
xv
6.13 Spike Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
164
7.1
Retinal Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
170
7.2
Chip Architecture and Layout . . . . . . . . . . . . . . . . . . . . . . . . . .
172
7.3
Spike Arbitration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
174
7.4
Chip Response to Drifting Sinusoid . . . . . . . . . . . . . . . . . . . . . . .
175
7.5
Luminance Adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
177
7.6
Chip Response to Drifting Sinusoids of Different Mean Intensities . . . . . .
180
7.7
Spatiotemporal filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
185
7.8
Changes in Open Loop Time Constant τna . . . . . . . . . . . . . . . . . . .
187
7.9
Changes in Open Loop Gain g
. . . . . . . . . . . . . . . . . . . . . . . . .
189
7.10 Contrast Gain Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
190
7.11 Change in Temporal Frequency Profiles with Contrast . . . . . . . . . . . .
193
7.12 Effect of WA Activity on Center Response . . . . . . . . . . . . . . . . . . .
195
xvi
Chapter 1
Introduction
The retina, an outgrowth of the brain that comprises ∼0.5% of the brain’s weight[99], is
an extraordinary piece of neural circuitry evolved to efficiently encode visual signals for
processing in higher cortical structures. The human retina contains roughly 100 million
photoreceptors at its input that transduce light into neural signals, and roughly 1.2 million
axons at its output that carry these signals to higher structures. Three steps define the
conversion of visual signals to a spike code interpretable by the nervous system: transduction
of light signals to neural signals, processing these neural signals to optimize information
content, and creation of an efficient spike code that can be relayed to cortical structures.
The retina has evolved separate pathways, specialized for encoding different features within
the visual scene, and nonlinear gain-control mechanisms, to adjust the filtering properties of
these pathways, in a complex structure that realizes these three steps in an efficient manner.
Although the processing that takes place in the retina represents a complex task for any
system to accomplish, the retina represents the best studied and best understood neural
system thus far.
1
Chapter 2 attempts to summarize the structure, function, and outputs of this complex
stage of visual preprocessing by dividing retinal anatomy into five general cell classes: three
feedforward cell classes and two classes of lateral elements. The three feedforward cell classes
— photoreceptors, bipolar cells, and ganglion cells — realize the underlying transformation
from light to an efficient spike code. The interaction between each of the feedforward cell
classes represents the two primary layers of the retina where visual processing takes place,
the outer plexiform layer (OPL) and the inner plexiform layer (IPL). The two lateral cell
classes — horizontal cells and amacrine cells — adjust feedforward communication at each
of these plexiform layers respectively. Understanding the synaptic interactions that underlie
this structure allows us to gain insight about the retina’s ability to efficiently capture visual
information.
The simplified description offered in Chapter 2 makes it clear that preprocessing of
visual information in the retina is significantly more complex upon closer inspection. Each
cell class, for example, does not represent a homogeneous population of neurons, but is
comprised of several types that are each distinguishable by their morphology, connections,
and function[85, 98]. These different cell types define the different specialized pathways the
retina uses to communicate visual information. Because of the complexity of the retina,
Chapter 2 attempts to emphasize only those elements within the retina that shed light on
how the mammalian retina processes visual information. It discusses how these different
cell types contribute to visual processing and summarizes the outputs of the retina and how
these outputs reflect visual processing. With this introduction to the retina, we can begin
to explore some of the properties of this processing scheme in order to both understand the
interactions that lead to this processing and to engineer a model that replicates the retina’s
behavior.
An anatomic description of the retina allows us to explore its organization, but to
2
fully understand the computations performed by the retina, we must study how the retina
responds to light and how it encodes this input in its output. To determine retinal function,
one can consider the retina a “black box” that receives inputs and generates specific outputs
for those inputs. The retina affords us a unique advantage in that its input, visual stimuli,
is clearly defined and easily manipulated. In addition, we can easily measure the retina’s
output by electrically recording ganglion cell responses to those visual stimuli. If we choose
the input appropriately, we can determine the function of the retina’s black box from this
input-output relationship.
Chapter 3 introduces a white noise analysis that attempts to get at the underpinnings
of how the retina processes information. Gaussian white noise stimuli are useful in determining a system’s properties because the stimulus explores the entire space of possible
inputs and produces a system characterization even if the presence of a nonlinearity in the
system precludes traditional linear system analysis. The white noise approach allows us
to deconstruct retinal processing into a simple model composed of a linear filter followed
by a static nonlinearity, and to explore how these components change in different stimulus
conditions. The simple model accounts for most of the ganglion cell response, and so exploring the parameters of that model allows us to understand how the retina changes its
computations across different cell types and how it adjusts its computations under different
stimulus conditions. Furthermore, the model allows us to explore discrepancies in retinal
processing found in different visual pathways, and therefore, to draw conclusions about the
importance of these specialized pathways in coding visual information.
Our understanding of the retina is based on the assumption that the retina attempts to
encode visual information as efficiently as possible. The retina communicates spikes through
the optic nerve, which presents a bottleneck through which the retina must efficiently send
important information about the visual scene. The anatomical review and physiological
3
explorations described in the first two chapters begin to characterize the retina’s efforts to
that end.
Chapter 4 introduces information theory as a different approach to understanding these
issues. This chapter adopts the information-theoretic approach to derive the optimal spatiotemporal filter for the retina and to make predictions as to how this filter changes as
the inputs to the retina change. To maximize information rates, the optimal retinal filter
whitens frequencies where signal power exceeds the noise, and attenuates regions where noise
power exceeds signal power. The filter thereby realizes linear gains in information rate by
passing larger bandwidths of useful signal while minimizing wasted channel capacity from
noisy frequencies. In addition, as inputs to the retina change, the retinal filter adjusts its
dynamics to maintain an optimum coding strategy. Chapter 4 provides a mathematical
description of this optimal filter and how it changes with input, and derives how processing
in the outer and inner retina might realize such efficient processing of visual information.
Because information theoretic considerations lead us to a mathematical expression for
the retina’s optimal filter and for how the retina adapts its filter to different input stimuli to
maximize information rates, we can explore how these adjustments are realized in response
to different conditions found in natural scenes. A goal of this approach is to quantify how
the retina adjusts its filters for different stimulus contrasts, and how the retina changes
its response to a specific stimulus when presented against a background of a much broader
visual scene. Furthermore, such conclusions require a description of the cellular mechanisms
underlying these adaptations and hypotheses for why the retina chooses these mechanisms
in particular.
Chapter 5 returns to the white noise analysis to explore these questions. Through the
linear impulse response and static nonlinearity characterized using the white noise analysis,
4
Chapter 5 directly examines how retinal filters change with different stimulus conditions.
The analysis focuses on the linear impulse response because it directly tells us how the retina
filters different temporal frequencies in the visual scene. The chapter examines the changes
in the ganglion cell’s linear impulse response as we increase stimulus contrast and compares
those changes to those observed when we introduce visual stimuli in the ganglion cell’s
periphery. This approach allows us to propose a simplified model that mediates adaptation
of the retinal filter, one local and one peripheral, and to explore the validity of this model
using pharmacological techniques.
One approach for merging retinal structure and function, and for incorporating the
dynamic adaptations predicted by an optimal filtering strategy, is to replicate retinal processing in a simplified model. Modeling has traditionally been used to gain insight into
how a given system realizes its computations. Efforts to duplicate neural processing take a
broad range of approaches, from neuro-inspiration, on the one end, to neuromorphing, on
the other. Neuro-inspired systems use traditional engineering building blocks and synthesis
methods to realize function. In contrast, neuromorphic systems use neural-like primitives
based on physiology, and connect these elements together based on anatomy. By modeling
both the anatomical interactions found in the retina and the specific functions of these
anatomical elements, we can understand why the retina has adopted its structure and how
this structure realizes the stages of visual processing particular to the retina.
Chapter 6 introduces an anatomically-based model for how the retina processes visual
information. The model replicates several features of retinal behavior, including bandpass
spatiotemporal filtering, luminance adaptation, and contrast gain control. Like the mammalian retina, the model uses five classes of neuronal elements — three feedforward elements
and two lateral elements that communicate at two plexiform layers — to divide visual processing into several parallel pathways, each of which efficiently captures specific features of
5
the visual scene. The goal of this approach is to understand the tradeoffs inherent in the
design of a neural circuit. While a simplified model facilitates our understanding of retinal
function, the model is forced to incorporate additional layers of complexity to realize the
fundamental features of retinal processing.
After introducing the underlying structure of a valid retinal model, Chapter 6 details
how we can implement such a model in silicon. Replicating neural systems in analog VLSI
generates a real-time model for these systems which we can adjust and explore to gain
further insight. In addition, engineering these systems in silicon demands consideration of
unanticipated constraints, such as space and power. The chapter provides mathematical
derivations for the circuits we use to implement the components of our model and details
how these circuits are connected based on the anatomical interactions found in the mammalian retina. Finally, because we understand both the underlying model and the circuit
implementation of this model, the chapter concludes by making predictions for the output
of this model that we can specifically test.
Finally, Chapter 7 describes a retinomorphic chip that implements the model proposed
and detailed in Chapter 6. The chip uses fundamental neural principles found in the retina
to process visual information through four parallel pathways. These pathways replicate
the behavior of the four ganglion cell types that represent most of the mammalian retina’s
output. In this silicon retina, coupled photodetectors (cf., cones) drive coupled lateral
elements (horizontal cells) that feed back negatively to cause luminance adaptation and
bandpass spatiotemporal filtering. Second order elements (bipolar cells) divide this contrast signal into ON and OFF components, which drive another class of narrow or wide
lateral elements (amacrine cells) that feed back negatively to cause contrast adaptation
and highpass temporal filtering. These filtered signals drive four types of output elements
(ganglion cells): ON and OFF mosaics of both densely tiled narrow-field elements that give
6
sustained responses and sparsely tiled wide-field elements that respond transiently. This
chapter describes our retinomorphic chip and shows that its four outputs compare favorably to the four corresponding retinal ganglion cell types in spatial scale, temporal response,
adaptation properties, and filtering characteristics.
7
Chapter 2
The Retina
The retina is an extraordinary piece of neural circuitry evolved to efficiently encode visual
signals for processing in higher cortical structures. The retina, an outgrowth of the brain
that comprises ∼0.5% of the brain’s weight[99], is a thin sheet of neural tissue lining the
back of the eye. Visual signals are converted by the retina into a neural image represented
by a complex spike code that is conveyed along the optic nerve to the rest of the nervous
system. The human retina contains roughly 100 million photoreceptors at its input that
transduce light into neural signals, and roughly 1.2 million axons at its output that carry
these signals to higher structures. Although the processing that takes place in the retina
represents a complex task for any system to accomplish, the retina represents the best
studied and best understood neural system thus far.
Three steps define the conversion of visual signals to a spike code interpretable by the
nervous system: transduction of light signals to neural signals, processing these neural
signals to optimize information content, and creation of an efficient spike code that can be
8
relayed to cortical structures. These three steps are realized in the retina by three classes
of cells that communicate in a feedforward fashion: photoreceptors represent the first stage
of visual processing and convert incident photons to neural signals, bipolar cells relay these
neural signals from the input to output stages of the retina while subjecting these signals to
several levels of preprocessing, and ganglion cells convert the neural signals to an efficient
spike code[36]. The interaction between each of these cell classes represents the two primary
layers of the retina where visual processing takes place, the outer plexiform layer (OPL) and
the inner plexiform layer (IPL). Synaptic connections between the feedforward cell classes,
as well as additional interactions between lateral elements, characterize each of these two
layers.
The apparently simply three step design that defines retinal processing is significantly
more complex upon closer inspection. The three feedforward cell classes and the lateral
elements present at each of the retina’s two plexiform layers together comprise a total of
five broad cell classes. Each class, however, does not represent a homogeneous population of
neurons, but is comprised of several types that are each distinguishable by their morphology,
connections, and function[85, 98]. In all, there are an estimated 80 different cell types found
in the retina[62, 98, 105], an extraordinarily large number for a system that at first glance
seems designed simply to convert light to spikes. There is, however, a certain amount of
logic to this degree of complexity — the retina uses the different cell types to construct
multiple neural representations of visual information, each capturing a unique piece of
information embedded in the visual scene, and conveys these representations through an
elegant architecture of parallel pathways. The retina uses different combinations of different
cell types, and thus uses different neural circuits, to capture these representations in an
efficient manner over a large range of light intensities.
This chapter attempts to summarize the structure, function, and outputs of this complex
9
stage of visual preprocessing. Because of the complexity of the retina, this chapter attempts
to emphasize only those elements within the retina that shed light on how the retina processes visual information. Furthermore, because a comparison of retinal structure across
species would demand an extensive review, this chapter focuses on mammalian retina. It
begins by providing an anatomical review of the different cell classes and types and how
these cell types are connected within the retina’s architecture. The chapter then discusses
how these different cell types contribute to visual processing by exploring how these cell
types realize their respective functions. Finally, the chapter concludes by summarizing the
outputs of the retina, how these outputs reflect some of the processing that takes place
within the retina, and how we can interpret these outputs to further understand the retina.
2.1
2.1.1
Retinal Structure
Cell Classes
The optics of the eye are designed to focus visual images on to the back of the eye where the
retina is located. The retina receives light input from the outside world and converts these
visual signals to a neural code that is conveyed to the rest of the brain. It accomplishes this
task using three feedforward, or relay, cell classes and two lateral cell classes that contribute
to retinal processing of this information. Anatomists have divided the architecture of the
retina into three layers that each contain the cell bodies of one of the feedforward cell
classes — an outer nuclear layer (ONL) that contains the photoreceptors, an inner nuclear
layer (INL) that contains the bipolar cells, and a ganglion cell layer (GCL) that contains
the ganglion cells. In addition, the interaction between these relay cells occurs within two
plexuses — the more peripheral plexus is called the outer plexiform layer (OPL) while the
10
more central plexus is called the inner plexiform layer (IPL).
Each plexiform layer thus contains an input and output from two successive relay neurons. Each plexus also contains cells from one lateral cell class that communicate with the
two relay neurons present in that plexus. A radial section through the retina is shown in
Figure 2.1. The flow of visual information begins at the top of the image, where light is
detected by photoreceptors in the outer nuclear layer. Neural signals emerging from the
outer nuclear layer are conveyed to the inner nuclear layer through synaptic interactions
in the outer plexiform layer. The inner plexiform layer contains the synaptic interactions
that relay signals from the inner nuclear layer to the ganglion cell layer. Finally, ganglion
cells convey the neural information that has been processed by the retina to the rest of the
brain, sending axons out the bottom of the image.
A schematic showing the different cell classes, shown in Figure 2.2, and their relative sizes
and connectivity provides a more accessible representation of the flow of visual information.
The five different cell classes represented in the schematic are photoreceptors, horizontal
cells, bipolar cells, amacrine cells, and ganglion cells. The first stage of visual processing,
transduction of light to neural signals, is realized by the photoreceptors. Photoreceptors
are divided into two types of neurons in most vertebrates, rods and cones. Their cell bodies
lie in the ONL, and drive synaptic interactions in the OPL. The lateral cell class found
in the OPL are the horizontal cells. They provide inhibition in the OPL, and play an
important role in light adaptation and shaping the spatiotemporal response of the retina.
Their cell bodies lie in the INL immediately below the OPL. Primates have two types of
horizontal cells, HI and HII. The second class of relay neurons are called bipolar cells and
convey signals from the OPL to the IPL. Their cell bodies lie in the middle of the INL.
Their dendrites extend to the OPL, while their axons synapse in the IPL. This bipolar
structure lends this class of neurons their name. Bipolar cells come in a variety of types,
11
Figure 2.1: Different Layers in the Retina
A radial section through the monkey retina 5mm from the fovea (reproduced from [99]). Light signals
focus on the top of the image, and visual information flows downward. Ch, choroid; OS, outer segments;
IS, inner segment; ONL, outer nuclear layer; CT, cone terminal; RT, rod terminal; OPL, outer plexiform
layer; INL, inner nuclear layer; IPL, inner plexiform layer; GCL, ganglion cell layer; B, bipolar cell; M,
Muller cell; H, horizontal cell A, amacrine cell; ME, Muller end feet; GON , ON ganglion cell, GOFF , OFF
ganglion cell.
12
depending on the extent of their dendritic field and whether they encode light or dark
signals. The lateral cell class in the IPL is the amacrine cells. Their cell bodies lie in the
INL just above the IPL, although some amacrine cells, called displaced amacrine cells, lie
in the ganglion cell layer. Although most of their function remains unknown, it has been
suggested that amacrine cells play a vital role in processing signals relayed between bipolar
cells and ganglion cells. There are more than 40 types of amacrine cells[62, 105], and any
attempt to review their different functions and morphologies would be inadequate. The
third, and final, class of relay neurons are the ganglion cells. These neurons represent the
sole output for the retina, and their cell bodies lie in the GCL. Ganglion cells communicate
information from the retina to the rest of the brain by sending action potentials down their
axons. There are several different type of ganglion cells, discussed below, each responsible
for capturing a different facet of visual information.
2.1.2
Outer Plexiform Layer Structure
The outer plexiform layer represents the region where the synaptic interactions between
photoreceptors, bipolar cells, and horizontal cells occur. Photoreceptor cell bodies lie in the
outer nuclear layer, while bipolar and horizontal cell bodies lie in the inner nuclear layer, as
demonstrated in Figure 2.2. To understand how the architecture underlying the synaptic
organization of these three cell classes leads to their functions, we can review some of the
their structural properties.
Photoreceptors represent the first neuron cell class in the cascade of visual information.
They are the most peripheral cell class in the retina and are found adjacent to the choroid
epithelium that lines the retina at the back of the eye. Photoreceptors, which are elongated,
come in two types, rods and cones, which divide the range of light intensity over which we
13
Figure 2.2: The Flow of Visual Information in the Retina
Schematic diagram representing the five different cell classes of the retina. Light focuses on the outer
segments of the photoreceptors. Synapses in the outer plexiform layer relay information from the photoreceptors to the bipolar cells. The lateral cell class at this plexiform layer, the horizontal cells, receives
excitation from the cone terminals and feeds back inhibition. Synapses in the inner plexiform layer relay
information from the bipolar cells to the ganglion cells. The lateral cell class at this plexiform layer, the
amacrine cells, modifies processing at this stage. Reproduced from [34]
14
can see into two regimes. Both types have an outer segment that contains about 900 discs
stacked perpendicular to the cell’s long axis, each of which is packed with the photopigment
rhodopsin (reviewed in [80]). Mitochondria fill the inner segment of each photoreceptor and
provide energy for the ion pumps needed for transduction. Because the retina attempts to
maximize outer segment density to attain the highest spatial resolution, the photoreceptor
somas often stack on top of one another, as shown in Figure 2.1.
Cones and rods, which fill 90% of the two dimensional plane at the outer retina[78],
are responsible for vision during daytime and nighttime, respectively. Cones only account
for 5% of the number of photoreceptors in humans, yet their apertures account for 40% of
the receptor area[99]. The center of the retina, the fovea, represents the region of highest
spatial acuity. Here, cones are so densely packed (∼200,000 cones/mm2 [25]) that rods
are completely excluded from this region. Since rods are responsible for night vision, this
architecture means that humans develop a blind spot in the fovea once light intensity falls.
This specialization is species dependent — cats, which need to retain vision at night, have a
ten-fold lower cone density in the central area and allow for the presence of rods there[109].
In addition to differing in their sensitivity to light intensity, cones and rods differ in
their spectral sensitivity. Mammals only have a single type of rod that has a peak spectral
sensitivity of 500 nm[99]. However, higher light intensities afford the retina the ability to
discriminate between different wavelengths to increase information. Hence, in humans there
are three types of cones, each with a different spectral sensitivity. “M” or green cones are
tuned to middle wavelengths, ∼550 nm, and comprise most of the cone mosaic[51]. “S” or
blue cones form a sparse, but regular mosaic, in the outer nuclear layer and have a peak
sensitivity to short wavelengths, ∼450 nm[29]. Finally, “L” or red cones respond to long
wavelengths, ∼570 nm, and are nearly identical to M cones[99].
15
Photoreceptor axons are short and their synapse in the outer plexiform layer is characterized by the presence of synaptic ribbons. The ribbon is a flat organelle anchored at
the presynaptic membrane to which several hundred vesicles are “docked” and ready for
release. This structure facilitates a rapid release of five to ten times more vesicles than
found at conventional synapses[73]. Both rods and cones employ synaptic ribbons for communication with invaginating processes of post-synaptic neurons. Rods use a single active
zone that typically contains four post-synaptic processes, a pair of horizontal cell processes
and a pair of bipolar dendrites[81]. A schematic of a typical rod’s synaptic structure, called
a tetrad, is shown in Figure 2.3. Horizontal cell processes penetrate deeply and lie near
the ribbon’s release site while bipolar processes terminate quite far from the release site.
Cones also employ the ribbon synapse, although they have multiple active zones that are
each penetrated by a pair of horizontal and one or two bipolar cells[57, 16].
In addition to the ribbon synapse, cone terminals form flat or basal contacts with bipolar
dendrites[57, 16]. The mechanism of transmitter release at this contact is as yet unidentified.
However, the ribbon synapses are occupied exclusively by ON bipolar dendrites while many
of the basal contacts are occupied by OFF bipolar dendrites[58]. Admittedly, this distinction
is not quite so simple since many ON bipolar dendrites have basal contacts[16], but it appears
that the synaptic difference may play a role in differences between ON and OFF signaling.
Horizontal cells, which receive synaptic input from the photoreceptor ribbon synapse
and which represent the lateral cell class of the OPL, have cell bodies that lie in the inner
nuclear layer adjacent to the OPL. Horizontal cells receive input from several photoreceptors
and electrically couple together through gap junctions. The extent of this coupling has
been found to be adjustable in lower vertebrates, such as the catfish, by a dopaminergic
interplexiform cell[90]. In primates, horizontal cells come in two types, a short-axon cell,
HI, and an axonless cell, HII. The former has thin dendrites that collect from a narrow field
16
Figure 2.3: Rod Ribbon Synapse
This schematic illustrates the ribbon synapse found in an orthogonal view of the rod terminal — many
of the same principles extend to the cone and bipolar terminals. The tetrad consists of a single ribbon,
two horizontal cell processes (hz) and two bipolar dendrites (b). Many vesicles (circles) are docked at
the ribbon, facilitating rapid release of a large amount of transmitter. From [81].
17
and couples weakly to its neighbors, while the latter has thick dendrites that collect from
a wide-field and couples strongly[106]. HI communicates with rods through its axon, while
HII communicates exclusively with cones[91], although the functional distinction between
the two types remains unclear.
The bipolar cells, the third cell class that synapses in the OPL, represents the second
stage of feedforward transmission of visual information and relays signals from the OPL to
the IPL. Their cell bodies lie in the middle of the inner nuclear layer. Bipolar cells collect
inputs in their dendrites at the rod and cone terminals and extend axons to synapse with
amacrine cells and ganglion cells in the IPL. Bipolar cells can be divided into several types,
depending on which photoreceptor they communicate with and on what types of signals
they relay. Rod bipolar cells communicate exclusively with rods, and they are part of a
separate rod circuit discussed in Section 2.1.4. Cone bipolar cells typically collect input
from 5-10 adjacent cones[22, 16].
Cone bipolar cells are actually divided into two types, ON and OFF, depending on
whether they are excited by light onset or offset. As mentioned above, ON bipolar cells
typically have invaginating dendrites while OFF bipolar cells typically form flat contacts
with the overlying cones. More importantly, however, these bipolar cells differ in the types
of glutamate receptors they express — OFF bipolar cells express the ionotropic GluR while
ON bipolar cells express the metabotropic mGluR (see Section 2.2.1). Furthermore, ON and
OFF bipolar cells differ in where their axonal projections terminate — OFF bipolar axons
terminate in the more peripheral laminae of the IPL while ON bipolar axons terminate in the
more proximal laminae. Differences in axonal projection within these laminae suggest that
there are actually several subtypes of bipolar cells within the broad ON/OFF distinction[62,
22, 13, 42]
18
2.1.3
Inner Plexiform Layer Structure
The inner plexiform layer represents the region where the synaptic interactions between
bipolar cells, amacrine cells, and ganglion cells occur. Amacrine cell bodies primarily lie in
the inner nuclear layer, but some displaced amacrine cells can be found alongside ganglion
cells in the ganglion cell layer, as demonstrated in Figure 2.2. To understand how the
architecture underlying the synaptic organization of these three cell classes leads to their
functions, we again review their structural properties.
The IPL, which is five times thicker than the OPL, has been divided by anatomists into
five layers of equal thickness called strata[48] and labeled S1, the most peripheral stratum, to
S5. This anatomical division has a functional correlate — bipolar cells ramifying in S1 and
S2 drive OFF responses while bipolar cells ramifying in S4 and S5 drive ON responses[44, 77].
Bipolar cells that synapse with ganglion cells in the middle layers, S2 to S4, drive ganglion
cells with ON/OFF responses. Hence, a simpler division has emerged, one that divides the
IPL into two sublamina, ON and OFF .
Bipolar terminals are also characterized by the presence of synaptic ribbons, but postsynaptic processes do not invaginate the presynaptic membrane as found in the outer plexiform layer[99]. Two post-synaptic elements line up on both sides of the active zone, forming
a dyad[36]. These post-synaptic elements can be any combination of amacrine and ganglion
cells. However, when one of these elements is an amacrine cell, its processes often feedback
to form a reciprocal synapse[15].
Amacrine cells, which synapse in the IPL, are characterized by their extreme diversity.
There are over 40 types of amacrine cells[62], and the distinctions between most of these
types is as yet mostly unclear. However, there are four general types of amacrine cells that
19
we can generally describe. The AII amacrine cell, which comprises 20% of the amacrine
cell population, collects exclusively from rod bipolar cells, and is discussed in Section 2.1.4.
A second type of amacrine cell collects inputs from cone bipolar cells, is characterized by
its narrow input field, and provides both feedback and feedforward synapses on to bipolar
cells and ganglion cells respectively[99]. A third type of amacrine cell is the mediumfield amacrine cell, the most famous of this type being the starburst amacrine cell which
associates with other starburst cells and provides cholinergic input on to ganglion cells[70,
72]. Finally, a wide-field amacrine cell represents the fourth general type of amacrine cell
that synapses in the IPL. These cells collect inputs over 500-1000 µm[26]. Furthermore,
these wide-field amacrine cells, unlike the rest of the retinal cells presynaptic to the ganglion
cells, communicate using action potentials and so can relay signals over long distances[28,
45].
The ganglion cells are the retina’s only means to communicate signals to the rest of the
cortex. Their cell bodies lie in the innermost retinal layer, the ganglion cell layer. Ganglion
cells collect inputs at their dendrites from synaptic interactions in the IPL and project their
axons down the optic nerve to the rest of the brain. In humans, the optic nerve has roughly
1.2 million axons, but this number varies across species, suggesting that the optic nerve
does not in fact present a “bottleneck” for visual information[99].
Anatomists have divided the ganglion cell class into three major types, α, β, and γ.
Although ganglion cells project to such regions as the suprachiasmatic nucleus and the
superior colliculus, most of the axons (60% in cat, 90% in primate) in the optic nerve
project to the dorsal lateral geniculate nucleus (which then projects to the visual cortex)[99]
suggesting that most of the ganglion cells are dedicated to visual processing. α cells have a
wide, sparse dendritic tree and are characterized by their transient response[21, 101]. β cells
have a narrow, bushy dendritic tree and are characterized by their sustained response[12].
20
The γ type of ganglion cells represents the remaining ganglion cell types, including those
that project to regions other than the geniculate and direction-selective ganglion cells.
The α/β distinction has an analogous classification in primates: the narrow-field β
ganglion cells are called midget cells while the wide-field α cells are called parasol cells
in primate. Midget cells are also called “P” cells since they project to the parvocellular
layer of the geniculate while parasol cells are also called “M” cells since they project to
the magnocellular layer of the geniculate[54]. In addition to the anatomical distinction,
physiologists have divided the ganglion cell class into different functional types, X, Y, and
W. These distinctions are discussed in Section 2.2.2. However, in general, the correlation
between structure and function has been established over several decades of research, and
interchanging these different names has become commonplace.
The many ganglion cell types present a wide diversity of methods to encode visual information. Each ganglion cell type, then, is responsible for creating a neural representation
of the visual scene that captures a unique component of visual information. Thus, the dendrites of each ganglion cell type tile the retina and are therefore capable of collecting inputs
from every point within the visual scene[99]. There is little overlap between the dendritic
trees of two adjacent ganglion cells of the same type, and so redundancy of information is
eliminated. This extraordinary structure enables the retina to convey information to the
cortex along several parallel information channels.
2.1.4
Structure of the Rod Pathway
Rods are responsible for vision at low luminance conditions. Hence, a separate pathway
by which rods can communicate these low intensity signals to the cortex has emerged.
Because at low intensities, every photon becomes significant, and because the retina must
21
pool several of these photons together to differentiate the signal from the noise, the rod
bipolar cell collects inputs from several rods in the OPL[27, 110]. Furthermore, every rod
synapse contacts at least two rod bipolar cells, exhibiting a divergence that is not present in
the cone pathway[100, 110]. The rod bipolar dendrite penetrates the rod photoreceptor and
senses vesicle release from the ribbon synapse with a glutamatergic receptor[99]. The rod
bipolar extends its axon to the IPL and synapses in the ON laminae on to the AII amacrine
cell.
The AII amacrine cell, whose cell body is located in the inner nuclear layer, communicates to two structures in the IPL — it forms gap junctions to the ON cone bipolar terminals
and inhibitory chemical synapses with the OFF bipolar cells. Thus, the AII amacrine cells,
upon depolarization from rod excitation, is able to simultaneously excite the ON cone pathways and inhibit the OFF cone pathways. The divergence in the rod pathway, first seen
at the bipolar dendrite, continues with the AII amacrine cell. The rod bipolar axons tile
without overlap, but the AII’s dendritic fields overlap significantly, thus amplifying the signal from one bipolar cell through divergence[100, 110]. The significance of the rod pathway
is related to the ability of the retina to encode signals over several decades of mean light
intensity and is discussed in Section 2.2.1.
2.2
2.2.1
Retinal Function
Outer Plexiform Layer Function
The first stage of visual processing entails transduction of optical images to neural signals,
and this process is realized by the photoreceptors that lie in the outer nuclear layer and
that synapse in the outer plexiform layer. The retina is capable of encoding light signals
22
that range over ten decades of intensity. No other sensory system exhibits this tremendous
dynamic range. The cones and rods are the two primary types of photoreceptors and they
divide this range into day and night vision respectively. Cones have an integration of time
of 50 msec and are able to produce graded signals that can code 100 to 105 photons per
integration time[99]. Rods have an integration time of 300 msec, and produce graded signals
that can only code up to 100 photons per integration time which allows it to continue graded
signaling at light intensities that fall below the cone threshold. Most of the rod activity,
however, is binary — it signals the presence of absence of a single photon.
Photons incident on the back of the eye are trapped by the cone inner segment which acts
as a “wave-guide” and funnels these photons to the outer segment where they transfer their
energy to a rhodopsin molecule[37]. Rods exhibit a similar kind of transduction, although
their inner segments do not act to funnel photons to their outer segments — photons
simply pass through the inner segment and excite rhodopsin in the outer segment[37]. The
activation (isomerization) of the rhodopsin molecule causes a drop in cGMP concentration,
which causes cation channels to close and causes the outer segment to hyperpolarize[80].
This hyperpolarization is relayed to the inner segment and reduces the level of quiescent
glutamate released from the photoreceptor’s synapse.
The difference in range over which rods and cones respond is a result of their respective
sensitivities. Thermal agitation causes random isomerization of the rhodopsin molecule that
produces a baseline dark current. In the rod, one photon activating one rhodopsin molecule
is capable of reducing the dark current by 4%[97]. Cones, on the other hand, are roughly 70
times less sensitive — one photon reduces the dark current by 0.06%, which is masked by
the noise of random fluctuations. It thus takes roughly 100 isomerized rhodopsin molecules
arriving simultaneously to produce a significant change in cone current[80].
23
This difference in sensitivity allows cones to capture a much larger dynamic range than
rods. However, the more sensitive rods are necessary to ensure vision in twilight and
starlight conditions. In the latter case, because rods are sensitive to even a single photon,
and because it would be difficult to distinguish between the drop in current from a single
photon versus thermal agitation, rods pool their inputs together on to the rod bipolar to
increase the signal to noise ratio[99]. Hence, the rod pathway sacrifices spatial acuity for
sensitivity, while the cone pathway sacrifices sensitivity to maintain spatial acuity. Under
twilight conditions, the rods are capable of encoding a graded signal up to 100 photons
per integration time, and so such pooling would be unnecessary. In this case, rods couple
to cones, providing them with the graded signal that cones are unable to encode at low
intensities[97].
Signals that reach the photoreceptor terminal are relayed to the cone bipolar cells
through a glutamatergic synapse. The ribbon synapses allow the rapid release of a large
number of glutamatergic vesicles, making signaling both more sensitive and less susceptible
to noise. Light causes cones to hyperpolarize, and thus decreases the glutamate release at
their terminals. As mentioned above, OFF bipolar cells express ionotropic GluR receptors
while ON cells express metabotropic mGluR receptors[99]. The former are sign preserving,
while the latter are sign reversing. Therefore, the onset of light causes a depolarization in
ON bipolar cells while the offset of light, which causes cones to depolarize, causes a depolarization in OFF bipolar cells. At the very first synapse of the visual pathway, the retina
has immediately divided the signal into two complementary channels. From a functional
standpoint, this is extremely efficient since each channel is capable of exerting its entire
dynamic range to encode its respective signals.
The cones larger dynamic range does not account for the retina’s ability to respond
over ten decades of mean light intensity. To handle this tremendous range, the cones shift
24
their sensitivity to match the mean luminance of the input[102]. This intensity adaptation
mechanism most likely involves the third cell class in the OPL, the horizontal cells. The
horizontal cells, which express gap junctions that enable them to electrically couple to
one another, average cone excitation over a large area. These cells express the inhibitory
transmitter GABA[19] and most likely provide feedback inhibition on to the cone terminals.
Bipolar dendrites thus receive input from the difference between the cone signal and its local
average, producing a response that is independent of mean intensity and whose redundancy
has been reduced.
The interaction between an inhibitory horizontal cell network and an excitatory cone
network does not only have implications for intensity adaptation, but helps shape the bipolar
cell’s response. One of these implications is the existence of surround inhibition in the cone
terminal response[4]. A central spot of light causes cones to hyperpolarize, but an annulus of
light causes the cone response to depolarize. This center-surround interaction is mediated by
the inhibitory horizontal cell networks, since the annulus of light will cause surround cones
to hyperpolarize, decreasing horizontal cell activity, and thus reducing GABA inhibition
on the central cone terminal. In addition, the interplay between the cone and horizontal
networks shapes the bipolar cell’s spatiotemporal profile, as will be discussed later in this
thesis.
Finally, the extent of horizontal coupling is not fixed, but seems to be affected by inputs
from interplexiform cells. Studies of this phenomenon have been limited to date, however
the general story emerging is that dopaminergic interplexiform cells modulate the extent
of horizontal cell coupling in response to changes in mean intensity[35, 76, 52] since the
ganglion cell receptive field has been found to expand in these low intensity conditions.
25
2.2.2
Inner Plexiform Layer Function
The inner plexiform layer represents the second stage of processing in the retina and converts
inputs from bipolar cells to several neural representations of the visual scene, captured by
a complex neural code, that are relayed out the retina and to the rest of the nervous
system. The most important synapse in the inner plexiform layer is the one between the
final two relay cell classes, the bipolar cells and the ganglion cells. Bipolar terminals release
glutamate from their synaptic ribbons and ganglion cells, which express GluR and NMDA
receptors[71], are therefore excited by bipolar cell activity.
Visual information is already divided into multiple channels, each representing a different neural image of the visual scene, before even reaching the ganglion cell layer. This
division is realized by the several different bipolar cell types and by the complementary signaling in ON and OFF channels that begins at the very first synapse in the visual synapse.
Each of these bipolar cell types feeds input to the different ganglion cell types discussed in
Section 2.1.3. These ganglion cell classifications, designated as α or parasol and β or midget,
represent the different ganglion cell morphologies. However, physiologists have also adopted
a different scheme to classify these ganglion cells based on their functional responses. These
cell types are called X-, Y-, and W-ganglion cells which are analogous to the α, β, and γ
anatomical classification. Thus, X-cells tend to have sustained responses and smaller receptive fields while Y-cells tend to have transient responses and larger receptive fields. The W
type includes all other types of ganglion cells, including edge-detector cells and direction
selective cells[99]. A schematic demonstrating the four major ganglion cell types is shown
in Figure 2.4. These four ganglion cell types carry most of the visual information to the
cortex in complementary ON and OFF channels.
In the distinction between α and β cells, the retina has decomposed visual information
26
Figure 2.4: Structure and Function of Major Ganglion Cell Types
β cells have a narrow dendritic tree, and thus a narrow receptive field, while α cells have a wide dendritic
tree. β cells respond to the onset or offset of light in a sustained manner while α cells produce a transient
response. Each type of ganglion cell, α and β, is further divided by their ON or OFF responses — ON cells
depolarize in response to light onset while OFF cells depolarize in response to light offset. Reproduced
from [87].
27
into two domains for efficient coding. α (or Y) cells tend to be very good at capturing low
spatial frequency and high temporal frequency signals while β (or X) cells tend to be very
good at capturing high spatial frequency and low temporal frequency signals. Thus, there is
a tradeoff between spatial and temporal resolution that is distributed between the retina’s
different output channels. The retina’s ability to use a parallel processing scheme improves
the efficiency of encoding visual information. With such a scheme, each channel can devote
its full capacity to encoding a particular feature of the visual scene. Presumably, the
brain interprets these simultaneous multiple representations to reconstruct relevant visual
information.
The distinction between X and Y cells however does not end at their spatiotemporal
profiles. Y cells are characterized by their frequency doubled responses to a contrast reversing grating — shifting the spatial phase of this grating fails to eliminate the second Fourier
component of the response[49]. X cells, on the other hand, exhibit no such nonlinearity.
This division of linear and nonlinear responses may also play an important role in motion
detection since the frequency doubled response means that Y cell responses would never be
eliminated in response to moving stimuli.
The interactions at the IPL are not quite as simple as a bipolar to ganglion cell feedforward relay of visual information. The lateral cell class present in this layer, the amacrine
cells, adjusts the interactions between bipolar cells and ganglion cells. Although there are
a great number of types of amacrine cells, most of the function is unknown and remains
speculative. Spiking wide-field amacrine cells may play a role in communicating information laterally over long ranges. Narrow-field amacrine cells have been hypothesized to play
an important role in such nonlinear retinal mechanisms like contrast gain control[107] (see
Section 2.3). AII amacrine cells clearly play a role in the rod pathway by conveying rod ON
bipolar excitation to ON bipolar cells. Beyond these examples, however, most amacrine cell
28
function remains unexplained.
2.3
Retinal Output
The retina produces multiple representations of the visual image to convey to higher cortical structures, but most of what we know about retinal processing has been discovered
through investigations of single retinal ganglion cells. Although such an approach is both
time-consuming and inadequate for explaining population coding, a tremendous amount
of information has been unveiled. The prevailing view of retinal processing is that visual
information is decomposed into two complementary channels, ON and OFF , that respond
to the onset or offset of light. This observation, first made by Barlow and Kuffler[4, 64],
marks the beginning of our attempts to decipher the retina.
Spots of light centered over a ganglion cell’s receptive field either increase of decrease
the cell’s firing rate, depending on the ganglion cell’s classification, ON or OFF . In addition,
however, stimuli in the ganglion cell’s receptive field surround cause an opposite effect on
the ganglion cell response. This phenomenon, termed surround inhibition, led Rodieck to
develop his influential model of retinal processing based on an excitatory center and an
inhibitory surround, which he termed the difference of Gaussian model[83]. This model
accounted for ganglion cell responses quite well, and although the model was modified to
include delays in the lateral transmission inhibitory surround signals, the general principle
still holds today.
The visual scene, of course, is not made up of simple spots and annuli, and with more
experience, physiologists developed stronger tools to elucidate retinal processing. One of
these tools was the use of the Fourier transform to determine how well ganglion cells respond
29
to different spatial and temporal frequencies. By stimulating the ganglion cell with a light
input modulated at a certain frequency, one can determine how receptive that ganglion cell’s
pathway is to that frequency by taking the Fourier transform of the response and calculating
the system’s gain for that frequency. Repeating this algorithm for several frequencies allows
us to construct a spatial and temporal profile of the ganglion cell response, and allows us
to explore how these profiles change with different stimulus conditions.
This new quantitative tool opened entirely new avenues of research. The retina provides
an ideal system for such a study since its inputs can be controlled and its outputs can be
easily recorded. With such a technique, physiologists have been able to map the response
profiles of both X and Y ganglion cells in cats[47] and to hypothesize why the retina dedicates
so much effort to making multiple neural representations of visual information.
Such an approach has allowed researchers to explore certain otherwise unattainable
aspects of retinal processing, like intensity adaptation, contrast gain control, and other
nonlinearities present in retinal processing. The retina has the unique ability to respond over
roughly ten decades of light intensity, a property unmatched by any other sensory system.
Its ability to accomplish this feat stems from its ability to adjust the dynamic range of its
outputs to the range of inputs[99]. Hence, ganglion cell responses to different input contrasts
remain identical across a broad range of intensity conditions[102]. Only by applying the
aforementioned quantitative techniques to determine the spatial and temporal profiles of
different retinal cell classes were modelers able to understand how the retina realizes such
adaptation. The second major nonlinearity found in retinal processing is contrast gain
control, first described by Victor and Shapley[93]. When presented with stimuli of higher
contrasts, ganglion cell responses become faster and less sensitive. An adequate model
explaining this phenomenon emerged again by resorting to these quantitative techniques.
This model supposes that a “neural measure of contrast,” which preferentially responds
30
to high input frequencies, adjusts the inner retina’s time constants[107]. It was the shift
to a more quantitative analysis that allowed both this mechanism to be explored and to
be explained. Finally, a third nonlinearity found in retinal processing, also discovered
through the use of these quantitative techniques, is nonlinear spatial summation in cat Y
cells, first described by Hochstein and Shapley[49]. This principle was elucidated by the
inability of the Y cell’s second Fourier component to be eliminated by a contrast reversing
grating, suggesting that certain nonlinear rectifying elements contribute to the ganglion cell
response. It was later found that these rectifying elements are the bipolar cells, that pool
their inputs on to the Y cell dendritic tree to generate the ganglion cell response[38, 31].
Thus, a description of retinal processing, based on quantitative measurements of single
ganglion cell responses, has emerged. This description is summarized in the model shown
in Figure 2.5. Light enters the system and is filtered in space by a modified difference of
Gaussian. The output at every spatial location, which should represent a contrast signal,
is bandpass filtered and rectified. The dynamics of this filter is adjusted instantaneously
by a contrast gain control mechanism whose input is the output of the rectified bandpass
response. Finally, the outputs at all spatial locations are pooled, passed through another
linear filter, and rectified to produce a spike output to send to the cortex. Such a model,
developed through the quantitative techniques discussed above, can predict ganglion cell responses quite well by changing the parameters of the model to account for different ganglion
cell types[75].
Recent studies have taken the quantitave analysis even further, to elucidate new unexplored mechanisms of retinal processing and to gain a better understanding of how the
retina combines its multiple neural representations to capture all aspects of visual information. Thus, a contrast adaptation mechanism, by which the retina adjusts its sensitivity to
different contrasts over a long time scale, has recently been elucidated[95]. Furthermore,
31
Figure 2.5: Quantitative Flow of Visual Information
Light input, I(x, t), is filtered by a modified difference of Gaussian spatial filter which produces a pure
contrast signal to convey to subsequent processing stages. The signal is bandpass filtered and rectified.
The dynamics of the bandpass filter are adjusted by a contrast signal, c(t), that depends on the rectified
output of the bandpass response. Finally, signals are pooled from several spatial locations and passed
through another stage of linear filtering and rectification to produce the spike response, R(t). Reproduced
from [75].
32
population studies have demonstrated the ability of the retina to maintain high temporal
precision across multiple ganglion cells[7]. In general, the trend has been to use more complicated quantitave techniques and more appropriate stimuli, like natural scenes and white
noise stimuli, to better approximate what the retina actually has evolved to encode, to gain
a better understanding of retinal processing.
2.4
Summary
This brief summary of the structures and function of the retina gives some insight to the
complexities underlying this neural system. Because the retina produces multiple representations of the visual scene, modeling these outputs becomes a difficult task. And because
these different pathways communicate with one another and alter their respective behaviors,
efforts to capture all the elements of retinal processing becomes that much more difficult.
Any attempt at this point to replicate retinal function would have to be based on a simplified structure that captures the main features found in the retina. The strategy outlined
in this thesis pursues one of these attempts and, although incomplete, captures most of
the relevant processing found in the mammalian retina. The strategy focuses on producing
a parallel representation of the visual scene through the retina’s four major output pathways, and on introducing nonlinearities such as contrast gain control and nonlinear spatial
summation to these pathways.
33
Chapter 3
White Noise Analysis
While understanding the anatomic structure of the retina allows us to explore its organization, to fully understand the computations performed by, and hence the purpose of, the
retina, we must study how the retina responds to light and how it encodes this input in
its output. Kuffler initiated this physiological approach to investigating the retina with his
classic studies that elucidated the ganglion cells’ center–surround properties[64]. Since Kuffler’s work, physiologists have unmasked a wealth of data detailing the precise computations
performed by the retina (for review, see [99]). Physiological studies get at the underpinnings of how the retina processes information and are a vital component of any attempt
to determine function. Such an understanding is necessary to construct viable models of
retinal processing.
One can determine the function of a system without knowing its precise mechanisms by
studying the input-output relationship of that system. Thus, to determine retinal function,
neurophysiologists consider the retina a “black box” that receives inputs and generates
34
specific outputs for those inputs. The retina affords us a unique advantage in that its
input, visual stimuli, is clearly defined and easily manipulated. In addition, we can easily
measure the retina’s output by electrically recording ganglion cell responses to those visual
stimuli. If we choose the input appropriately, we can determine the function of the retina’s
black box from this input-output relationship. In this section, we present a white noise
approach for determining the retina’s input-output relationship. Such an approach allows
us to deconstruct retinal processing into a linear and nonlinear component, and to explore
how these components change in different stimulus conditions.
3.1
White Noise Analysis
Most descriptions of the retina’s stimulus–response behavior have been qualitative in nature or limited to spots and gratings — classic stimuli that give a limited quantitative
description of receptive field organization and spatial and temporal frequency sensitivity.
More recently, however, neurophysiologists have taken advantage of Gaussian white noise
stimuli to generate a complete quantitative description of retinal processing[69, 89, 17, 56].
Gaussian white noise is useful in determining a system’s properties because this stimulus
explores the entire space of possible inputs and produces a system characterization even
if a nonlinearity is present in the system, which precludes traditional linear system analysis. Gaussian white noise has a flat power spectrum and has independent values at every
location, at every moment, that are normally distributed. The stimulus thus represents
a continuous set of independent identically distributed random numbers with maximum
entropy.
Drawing conclusions from the retina’s input-output relationship using a white noise
stimulus requires us to model that relationship with a precise mathematical description. We
35
conceptualize the functions underlying retinal processing with this model. A simple linearnonlinear model for the retina’s input-output behavior[63], shown in Figure 3.1, assumes
the black box contains a purely linear filter followed by a static nonlinearity. A linear
kernel, h(t), filters inputs to the retina, x(t), producing a purely linear representation
of visual inputs, y(t). Such linear filtering is easy to conceptualize because it obeys the
principles of superposition and proportionality. A static nonlinearity subsequently acts on
y(t) to produce the retinal output, z(t). By characterizing this nonlinearity, we can quantify
exactly how retinal responses deviate from linearity.
The parameters of the linear-nonlinear model in Figure 3.1 represent a solution for how
the retina processes input, but it is not a unique solution. In theory, several combinations
of linear kernels (also called the impulse response), h(t), and static nonlinearities can be
combined to produce the same retinal output z(t) for a given input x(t). To understand
this property, we express the output of the system as a function of the input, x(t):
z(t) = N (x(t) ∗ h(t))
where N () represents the static nonlinearity and where ∗ represents a convolution. We can
see how this solution is not unique by dividing the impulse response, h(t), by a gain, ζ.
Since convolution is a linear step, we can pull this term outside the convolution:
1
z(t) = N x(t) ∗ h(t) = N
ζ
1
(x(t) ∗ h(t))
ζ
We can compensate for this attenuation by simply incorporating the same gain, ζ, into the
static nonlinearity, N (), to restore the original response z(t). Thus, multiple linear filters
36
Figure 3.1: Linear-Nonlinear Model for Retinal Processing
Computations within the retina are approximated by a single linear stage with impulse response h(t) that
produces an output y(t) for input x(t) and a single static nonlinearity that converts y(t) to the ganglion
cell response z(t).
and static nonlinearities that relate to one another through such scaling yield solutions for
our system. Because of the non-uniqueness of the solutions, we have the liberty to change
both the linear impulse response and static nonlinear filter without changing how the overall
filter computes retinal response. This means that if we want to explore how the impulse
response changes across conditions, for example, we can scale the static nonlinearities of
these conditions so that they are identical and then compare the impulse responses directly
after scaling them appropriately. This also implies that the linear filter and static nonlinearity do not uniquely reflect processing in the retina; they simply provide a quantitative
model from which we can draw conclusions about retinal processing.
In order to quantify the mammalian retina’s behavior, we recorded intracellular membrane potentials from guinea pig retinal ganglion cells (for experimental details, see Appendix A). Following a strategy similar to that used by Marmarelis[68], we presented a
Gaussian white noise stimulus to the retina and recorded ganglion cell responses. We presented the white noise stimuli as a 500µm central spot whose intensity was drawn randomly
from a Gaussian distribution every frame update. The standard deviation, σ, of the distribution defined the temporal contrast, ct, of the stimulus. Unless otherwise noted, we
presented stimuli for two minutes and recorded responses as discussed above.
37
For an ideal white noise stimulus, x(t), each value represents an independent identically
distributed random number. We ignore stimulus intensity since the retina should maintain
the same contrast sensitivity over several decades of mean luminance[102], and so the white
noise stimulus, x(t), that we use in our derivation has zero mean. Thus, the autocorrelation
of a white noise stimulus is:
φxx (τ ) = E[[x(τ )x(t − τ )]]
= P δ(τ )
(3.1)
(3.2)
where E[[..]] denotes expected value, P represents the stimulus power, and δ(τ ) is the Dirac
delta function. For a Gaussian white noise stimulus with standard deviation σ, the power
P equals the variance, σ 2 . Hence, our visual stimulus, with temporal contrast ct ≡ σ, has
power ct2 .
An input white noise stimulus x(t) evokes the typical ganglion cell response z(t) shown
in Figure 3.2. We recorded ganglion cell membrane potential and spike trains in response
to two minutes of white noise stimulus. The first twenty seconds of response were discarded
to permit contrast adaptation to approach steady state[56, 17]. To determine the system’s
linear filter, we cross-correlate the ganglion cell output with the input signal. For the
membrane response, the cross-correlation is straightforward, as the ganglion cell response
is simply a vector of values — the intracellular voltage in millivolts — sampled every
millisecond. In addition, we subtract out the resting potential, measured by averaging
the intracellular voltage for five seconds before and five seconds after introduction of the
stimulus, to get a zero-mean response vector. For spikes, we convert the spike train to
another vector of responses, also sampled every millisecond. In this case, however, every
sample in the vector takes an arbitrary value of 1 or 0, depending on the presence or absence
38
of a spike at that particular sample time. Cross-correlating these response vectors with the
input yields:
φxz (ψ) = E[[(x(t)z(t + ψ)]]
(3.3)
where we express the cross-correlation as a function of a new variable, ψ. Since we are
initially interested in finding the system’s linear component, we can, for the moment, ignore
nonlinearities in the system and express the output z(t) as the convolution of input x(t)
and linear filter h(t). In addition, we assume the system to be causal, so we integrate from
zero to infinity. Equation 3.3 becomes
Z ∞
φxz (ψ) = E[[
h(τ )x(t + ψ − τ )x(t)dτ ]]
(3.4)
0
We can interchange the integral and the expected value to solve for the linear component
h(ψ). Hence,
Z ∞
φxz (ψ) =
Z0∞
=
h(τ )E[[x(t)x(t + ψ − τ )]]dτ
(3.5)
h(τ )φxx (τ − ψ)dτ
(3.6)
0
From Equation 3.2, we know that the autocorrelation of the white noise stimulus yields an
impulse. Thus, the linear filter is given as:
Z ∞
φxz (ψ) =
h(τ )P δ(τ − ψ)dτ
0
39
(3.7)
Figure 3.2: White Noise Response and Impulse Response
A 500µm central spot whose intensity was drawn randomly from a Gaussian white noise distribution,
updated every 1/60 seconds, evokes a typical ganglion cell response (lower left) when presented for
two minutes. Cross-correlation between the membrane potential and the stimulus yields the membrane
impulse response (top right) and cross-correlation between the spikes and the stimulus yields the spike
triggered average (bottom right).
= P h(ψ)
(3.8)
The cross-correlation we compute from our recordings is in units of mV·ct for the membrane
response and units of S·ct for the spike response, where S represents an arbitrary unit. To
generate a membrane impulse response in units of mV/ct, or S/ct for spikes, we normalize
the impulse response h(ψ) by signal power σ 2 . Thus, the impulse response, or the purely
linear filter, of the retina is
1
φxz
=
h(t) =
P
P
Z
eiωt
X(ω)Z ∗ (ω)dω
2π
40
(3.9)
where φxz is the cross-correlation we compute from our direct measurements. The second
part of Equation 3.9 relates our analysis to an alternative approach for computing the
impulse response h(t), used in previous studies[56]. Here, X(ω) is the Fourier transform
of the white noise stimulus x(t) and is given by X(ω) =
R −iωt
e
x(t)dt and Z ∗ (ω) is the
complex conjugate of Z(ω), the Fourier transform of the output z(t). The two approaches
are equivalent. Thus, by cross-correlating either the membrane response or spike response
with the white noise input, we can derive both the membrane and spike linear filter h(t) in
Figure 3.1. h(t) is the system’s first-order kernel and is equivalent to the system’s impulse
response.
We can compute a linear prediction, in units of mV or in arbitrary units of S, of the
response of the cell, y(t), by convolving the linear filter h(t) with the stimulus x(t):
Z ∞
y(t) =
h(τ )x(t − τ )dτ
(3.10)
0
The linear predictions computed for both the membrane and spike impulse responses are
shown in Figure 3.3. These predictions represent the retina’s output if the system’s responses were purely linear. In practice, however, the retina exhibits nonlinearities in its
response. Our model assumes that we approximate these nonlinearities with a static nonlinearity, N (). To determine the parameters of the static nonlinearity, we can compare the
linear prediction to the measured response at every single time point. The two minute white
noise stimulus, sampled every millisecond, produces 120,000 such time points, and mapping
this comparison for every point of prediction and response produces a noisy trace. Instead,
we calculate the average measured response for time points that have roughly the same
value in the linear prediction. We mapped out the static nonlinearity this way, where the
average of similarly valued points in the linear prediction determined the x-coordinate and
41
Figure 3.3: System Linear Predictions
Membrane and spike impulse responses can be convolved with the white noise stimulus to yield a linear
prediction of the ganglion cell’s response.
42
the average measured response for those values determined the y-coordinate. We were able
to compute static nonlinearities for the transformation from membrane linear prediction to
membrane response and for spike linear prediction to spike rate. The static nonlinearity
for membrane response, shown in Figure 3.4, illustrate this mapping for one cell. The spike
static nonlinearity for the same cell is shown in Figure 3.5. The circles represent the average
measured response of 3200 similarly valued points in the linear prediction. Error bars in
the figure represent the SEM of these 3200 measured values. If the cell responded linearly
to light, we would expect the points to lie on a straight line. Instead, the shape of the curve
clearly deviates from linearity for both membrane potential and spike rate.
To quantify the shape of this nonlinearity, N (), we fit the points with a cumulative
normal distribution function, which provides an excellent fit to the static nonlinearity:
N (x) = αC(βx + γ)
(3.11)
where α, β, and γ represent the max, slope, and offset of the cumulative distribution
function, C(x). The fit is shown with the static nonlinearities as the solid line in Figures 3.4
and 3.5. Since the use of a cumulative distribution function, N (), is an arbitrary choice
we made because of how well it fits the nonlinearity, the use of any other smooth function
with interpretable parameters would also provide an equally valid description of the static
nonlinearity.
The model shown in Figure 3.1 captures most of the structure of the ganglion cell’s light
response. We can predict the response of a cell, z(t), to continuously varying light stimulus
x(t) by passing x(t) through the linear kernel, h(t), and passing the output of the filter
through the static nonlinearity, N ():
43
Figure 3.4: Mapping Static Nonlinearities
The static nonlinearity for membrane response is shown on the top right and illustrates how the linear
membrane prediction (bottom, rotated 90◦ ) compares to the recorded membrane potential (left). Every
point on the graph represents the average mapping of 3200 similarly valued points in the linear prediction.
Error bars represent SEM of the membrane response these points map to. The solid trace shown with
the static nonlinearity is a cumulative normal distribution function fitted to the individual data points.
44
Figure 3.5: Spike Static Nonlinearity
The static nonlinearity for spike response illustrates how the linear spike prediction compares to the
recorded spike rates. Every point on the graph represents the average mapping of 3200 similarly valued
points in the linear prediction. Error bars represent SEM of the spike rate these points map to. The
solid trace shown with the static nonlinearity is a cumulative normal distribution function fitted to the
individual data points.
Z
z(t) = N
(x(τ )h(t − τ )dτ
(3.12)
To verify that the parameters of the linear filter, h(t), and of the static nonlinearity, N (),
account for most of the ganglion cell’s response, we repeated a five second 500µm white
noise sequence twenty times. The individual trial membrane and spike responses are shown
in Figure 3.6. To get an estimate for how reliable the cell’s responses were, we averaged the
response for nineteen of the twenty trials and compared this average to the responses from
one trial. In addition, to generate our model’s predicted response, we convolved the same
five second white noise sequence with the linear filter, h(t), and passed the output through
the static nonlinearity, N(). If the linear-nonlinear model were accurate, simply knowing
the parameters of this model will allow us to predict the cell’s responses as well as we would
have predicted it using the average of the nineteen other trials. In fact, we found that the
45
root-mean-squared (RMS) error for the model’s prediction were statistically similar to the
RMS error for the prediction based on the average response. For the membrane response,
the model yielded an average RMS error of 1.75±0.31 mV while the prediction based on the
average response yielded an average RMS error of 1.43±0.26 mV. For the spike response,
the model yielded an average RMS error of 0.43±0.07 sp/bin (binsize is 1/60 seconds) while
the prediction based on the average response yielded an average RMS error of 0.333±0.05
sp/bin. A plot showing the RMS error from the average response’s prediction versus the
RMS error from the model’s prediction is also shown in Figure 3.6 for five OFF and three
ON cells.
While the system’s impulse response is easy to conceptualize because of the principles
of linearity, the static nonlinearity is less straightforward. The shape of the membrane
nonlinearity represents how nonlinear the inputs to the ganglion cell are, while the spike
nonlinearity incorporates both input nonlinearities and nonlinearities associated with the
cell’s spike generating mechanism. Hence, the membrane nonlinearity represents how the
retina transforms its inputs into ganglion cell membrane voltages while the spike nonlinearity
measures how the retina transforms its inputs into ganglion cell spikes. To measure these
input-output curves directly for the ganglion cell, as a control, we presented a 500µm spot
of different contrast levels to the retina and recorded the intracellular ganglion cell response
(Figure 3.7). We presented each flash of light at a given contrast level for one frame
(∼17 msec) followed by 59 frames of mean intensity, repeated for five seconds. Stimuli
were defined by Michelson contrast ((Istim − Imean )/Imean ), where Istim and Imean are the
stimulus and mean intensity.
The raw intracellular response to one of these flashes of light at five different contrasts
is shown in Figure 3.7b, left. We computed the membrane voltage, Vm, and spike rate,
Sp, for each response (Figure 3.7b, left). We averaged these responses over the five trials
46
Figure 3.6: Predicting the White Noise Response
Ganglion cell response to a five second sequence of white noise stimulus repeated twenty times. Spike
and membrane rasters and histograms are shown on the left for a typical cell. Below each histogram, the
raw data from one response, the averaged response from the remaining nineteen trials, and the model
prediction are shown for both spike rate and membrane potential. On the right, a comparison of RMS
errors from the data prediction versus the model prediction are shown for five OFF and three ON cells for
both spike rate and membrane potential. Binsize is 1/60 seconds.
47
for a given contrast level. The averaged membrane response to the same five contrasts
is shown in Figure 3.7b (center). The average spike rate response to the flash of light is
shown in Figure 3.7b (right). The ganglion cell responses looked asymmetric — depolarizing
responses to the preferred contrasts for a given cell (light on for ON cells, light off for OFF
cells) were larger than hyperpolarizing responses to the opposite contrasts. To quantify
this asymmetry in the membrane response, we averaged the ganglion cell’s intracellular
potential at a specific time point during the response. The time point was determined by
finding when the cell’s membrane potential first exceeded 75% of its maximum response to
a 100% contrast flash of its preferred sign. We chose this time point because it represented
the purely linear drive of the cell - contrast gain control and other saturating nonlinearities
had not appeared in the flash response by this time. The membrane potential at this time
point, in response to flashes of different contrast, is shown in Figure 3.7c, left and the
average spike rate at this same time point is shown in Figure 3.7c, right, for both and OFF
and ON cell (top and bottom respectively). The asymmetry in the ganglion cell’s flash
response and the static nonlinearity computed from the white noise analysis appeared to
be qualitatively very similar, confirming that the static nonlinearity indeed represents the
cell’s input-output curve.
3.2
On-Off Differences
From Figure 3.7c, we see that ON and OFF cells differ in their input-output curves. Both the
membrane potential and the spike rate exhibit a rectifying nonlinearity for OFF cells. ON
cell membrane potential responses, however, are much more linear than OFF cells and do not
exhibit this extreme rectification. This suggests ON and OFF cells differ in the parameters
that govern their respective system models, and hence in the mechanisms that underly the
48
Figure 3.7: Ganglion Cell Responses to Light Flashes
(a) A 500µm central spot presented over the dendritic field of a typical ganglion cell for ∼17 msec at
different contrast levels evokes the responses shown in (b). In (b), raw recordings of the intracellular
voltage in response to a 100% contrast flash is shown in the left column on top, and the extracted Vm
and Sp responses are shown in the middle and bottom left, respectively. Membrane potentials (spikes
clipped as in [30]) and spike rates are shown in the middle and right columns respectively, averaged over
five trials, at different contrast levels. (c) The average deviation of the membrane potential from rest
(left) and the spike rate (right) at a given time point (see text) is plotted for flashes of different contrasts
for an OFF (top) and an ON (bottom) cell. Error bars represent SEM. Contrasts, plotted on the x-axis,
correspond to deviations from mean luminance of the preferred sign (light on for ON cells, light off for
OFF cells). Note that the ON cell had a linear contrast response curve while the OFF cell exhibited a
strong rectification in response to negative contrasts (light on). The solid trace shown with the flash
response is a cumulative normal distribution function fitted to the individual data points.
49
computations they perform.
To quantify the differences between ON and OFF cells, we computed the impulse response and static nonlinearity for nineteen OFF cells and ten ON cells. Normalized impulse
responses for typical ON and OFF cell are shown in Figure 3.8 for both membrane and
spikes. The linear responses for ON and OFF cells look remarkably similar to one another, although their signs are reversed. This similarity holds for both membrane and spike
impulse responses, although the spike impulse response seems to precede the membrane
impulse response in both ON and OFF cells. We averaged the normalized membrane and
spike impulse responses for all nineteen OFF and all ten ON cells and plotted them on the
same graph for comparison (Figure 3.8 bottom). The impulse responses show remarkable
consistency between cells of a given type, and the average ON and OFF kernels are virtually
mirror images of one another.
We verified this symmetry between linear ON and OFF kernels by measuring the peak,
zero, and undershoot times for the impulse response of each cell. The results of these
measurements are shown in Figure 3.9a. The qualitative similarity that we observe between
ON and OFF linear kernels in Figure 3.8 is verified by the similarity of these three time points
between ON and OFF cells. We also compared the peak time between membrane and spike
impulse responses to confirm that the spike response peaked earlier. For all thirty cells,
the membrane peak time was delayed by 12.6 msec on average, as shown in Figure 3.9b.
To further quantitate the difference between ON and OFF linear kernels, we also measured
the amplitude of the normalized impulse response’s undershoot (Figure 3.9c). The extent
of the impulse response’s undershoot reflects how much the system temporally bandpass
filters input signals. ON and OFF undershoot amplitudes were similar for both membrane
and spike kernels, suggesting that both ON and OFF linear filters are similar.
50
Figure 3.8: Normalized Impulse Responses
Linear impulse responses generated by the white noise analysis in response to a two minute sequence of
white noise stimulus. Typical membrane and spike impulse responses for an ON and OFF cell are shown
on top, and the average impulse response of nineteen OFF and ten ON cells is shown on bottom. Shaded
regions represent SEM.
51
Figure 3.9: Impulse Response Timing
(a) The time point corresponding to the peak, zero-crossing, and undershoot is calculated for every
membrane and spike impulse response from Figure 3.8. The average of these time points for both ON
and OFF cells is represented in the bar graph on the right. Error bars represent SEM. (b) Spike impulse
response peak times precede membrane impulse response peak times by an average of 12.6 msec. (c)
The peak of the negative lobe in the impulse response is recorded for all cells. The average response
for both membrane and spike impulse responses is shown on the right for ON and OFF cells. Error bars
represent SEM.
52
Because we found that the linear impulse response of ON and OFF cells are similar,
we explored differences between the static nonlinearities of the two cell types that might
account for the differences in their input-output relationship demonstrated in Figure 3.7c.
Static nonlinearities normalized to the peak value are shown for a typical ON and OFF
cell in Figure 3.10 for both membrane and spikes. Like the ganglion cell flash response,
OFF cell membrane static nonlinearities deviate from linearity for negative values of the
linear prediction. ON cells, however, tend to remain linear in their membrane response.
As expected, both ON and OFF cells exhibit a strong rectifying nonlinearity in their spike
response, but ON cells tend to have a lower spike threshold than OFF cells. We averaged
the normalized membrane and spike static nonlinearities for all nineteen OFF and ten ON
cells and plotted them on the same graph for comparison (Figure 3.10, bottom). Like
the impulse response, the static nonlinearity demonstrates remarkable consistency across
cells of a given type. Unlike the impulse response, however, the membrane and spike
static nonlinearities exhibit consistent differences between ON and OFF cells — the ON cell
membrane nonlinearity tends to be more linear while the OFF cell spike nonlinearity tends
to have a higher spike threshold.
We confirmed these differences between ON and OFF static nonlinearities by quantifying
their deviations from linearity. We call our metric that we use to quantify these differences
the static nonlinearity index (SNL index). As shown in Figure 3.11a, we compute the slope
of the plosive side of the static nonlinearity at a point that lies at 75% of the maximum
value of the linear prediction (a=slope0.75 ). We similarly compute the slope on the negative
side (b=slope−0.75 ). Our SNL index is simply the log of the ratio between the two slopes
(SNL index = log10 (a/b)). The index thus represents how symmetric the curve is in the
positive and negative directions, and hence how rectified the static nonlinearity is. If the
system were completely linear, the static nonlinearity would have an SNL index of zero
since the two slopes would be the same. As the static nonlinearity becomes more rectified,
53
Figure 3.10: Normalized Static Nonlinearities
Normalized membrane and spike static nonlinearities as computed by the white noise analysis are shown
for a typical ON and OFF cell (top). The average normalized membrane and spike static nonlinearities
are shown for all ten ON and nineteen OFF cells. Shaded regions represent SEM.
54
or more asymmetric, the SNL index rises above zero. We compared the SNL indices for
membrane and spike nonlinearities between ON and OFF cells and found that OFF cells
had a larger SNL index for both membrane and spikes. This confirms that the OFF static
nonlinearity is more rectified. For every cell recorded, we measured the membrane and spike
SNL index to produce the scatter plot shown in Figure 3.11b. From the figure, we see that
OFF cells fall into a distribution with a greater membrane and spike SNL index than the
ON cell distribution. Furthermore, the membrane SNL index is correlated with the spike
SNL index for all cells (r=0.73).
The difference in SNL index between ON and OFF cells suggests that the synaptic inputs
driving these ganglion cell types are different. Earlier studies have demonstrated that the
rectified nonlinear subunits that converge to drive ganglion cell responses can be accounted
for by rectified bipolar inputs[31, 38]. Hence, exploring how the input-output curves differ
between ON and OFF cells yields some insight as to how the bipolar inputs to these cells
differ. To show that the input-output curves indeed differed between ON and OFF cells,
and to verify that the difference in SNL index is reflective of the difference in these curves,
we recorded the input-output curve for both cell types in response to a brief 500µm spot
of different contrast levels. As discussed above, we presented each flash of light at a given
contrast level for one frame (1/60 seconds) followed by 59 frames of mean intensity, repeated
for five seconds. We recorded the peak membrane voltage and spike rate in response to the
flash of light for seven ON and ten OFF cells. The peak responses, normalized to the highest
contrast level, are shown in Figure 3.12 (mean and SEM). From the figure, we see ON and
OFF responses that are very similar to the SNL curves of Figure 3.10. OFF cell membrane
responses are more rectified than ON cell membrane responses, and OFF cells exhibit a
higher spike threshold.
In addition to differences in their excitatory input and spike generating mechanisms,
55
Figure 3.11: Static Nonlinearity Index
(a) To quantify the differences in rectification between ON and OFF cells, we compute a static nonlinearity
index (SNL index). We measure the slope of the static nonlinearity at points that lie at ±75% of the
maximum value of the normalized linear prediction. SNL index is equal to the log of the ratio between
these two slopes. The average SNL index for membrane and spike nonlinearities is represented in the
bar graph for both ON and OFF cells (error bars represent SEM). (b) Membrane SNL indices are plotted
versus spike SNL indices from the same cell. The correlation coefficient between membrane and spike
SNL index is 0.73 for all cells. OFF cells tend to have a higher SNL index than ON cells.
56
Figure 3.12: Normalized Vm and Sp Flash Responses
The normalized membrane (left) and spike (right) response is plotted for flashes of different contrasts
for ON and OFF cells. Data points represent mean normalized response and error bars represent SEM.
Contrasts, plotted on the x-axis, correspond to deviations from mean luminance in the preferred direction
(light on for ON cells, light off for OFF cells). OFF cells exhibit a stronger rectification in their membrane
responses than ON cells. In addition, OFF cells have a higher spike threshold than ON cells.
57
ON and OFF cells differ in how they receive inhibition. In response to a step of light of the
preferred sign (light on for ON cells, and light off for OFF cells), both ON and OFF cells
depolarize through direct excitation from bipolar cell glutamate release. However, a step
input in the opposite direction hyperpolarizes OFF cells directly and hyperpolarizes ON cells
indirectly. To demonstrate this effect, we stimulated ON and OFF cells with a 500µm spot
centered on the cell’s receptive field whose intensity was modulated with a 1Hz square wave
and recorded the intracellular potential while current clamping the cell at, above, and below
the resting potential (Figure 3.13a). Depolarizing cells attenuates the excitatory response
while hyperpolarizing the cells amplifies it, confirming that bipolar excitation is direct. In
OFF cells, depolarization increases the inhibitory response, also confirming that OFF cell
inhibition is direct. However, depolarization in ON cells decreases their inhibitory response,
suggesting that ON cell inhibition is indirect. By measuring the change in magnitude of the
inhibitory response in both ON and OFF cells, we found that OFF cell inhibition reverses
near -100mV while ON cell inhibition reverses near -30mV (Figure 3.13b), confirming that
ON cell inhibition is in fact indirect.
Our preliminary data suggests that there is some element of cross-talk between ON
and OFF pathways that contribute to their inhibitory responses. This is consistent with
earlier findings of vertical inhibition between ON and OFF laminae in the inner plexiform
layer[86]. When we applied L-2-amino-4-phosphonobutyrate (L-AP4), a metabotropic glutamate receptor competitive agonist that terminates ON bipolar input in ∼30 seconds, to
the superfusate, we found that the direct inhibition in OFF cells was eliminated. This
suggests that OFF cell direct inhibition is mediated through the ON pathway.
Differences in quiescent glutamate release from bipolar inputs can account for the differences in membrane response and in inhibition between ON and OFF cells. Bipolar excitatory
synapses are co-spatial with ganglion cells’ receptive fields[46, 23] and thus mediate the gan-
58
Figure 3.13: ON and OFF Ganglion Cell Step Responses
(a) We recorded ON (left) and OFF (right) ganglion cell responses to a 1Hz square-wave modulated
500µm central spot while holding the cell above and below resting potential. Depolarizing ganglion cells
causes both ON and OFF excitatory responses to decrease but only causes OFF inhibitory responses to
increase. ON ganglion cell inhibitory responses decrease when the cell is depolarized. (b) Plotting reversal
potentials for the excitatory and inhibitory components of the ganglion cell step response reveals that
while both excitatory drives reverse near zero, only the OFF cells exhibit direct inhibition.
59
glion cells excitatory response. Our data suggests that OFF cells receive inputs from bipolar
cells that have lower rates of baseline glutamate release. Depolarization of these bipolar
cells causes an increase in glutamate release, as expected, but hyperpolarization can only
reduce the already low glutamate release so far. Hence, negative inputs are rectified. ON
cells, however, could receive inputs from bipolar cells which have higher rates of baseline
glutamate release. Changing bipolar activity translates to roughly linear changes in the
rate of glutamate release, and therefore, to a more linear ON ganglion cell response.
With such an elevated release rate, ON bipolar cells can cause indirect inhibition simply
by reducing the glutamate released from their terminals. Whereas OFF cells, with their
low glutamatergic inputs, need direct inhibition to implement a hyperpolarization, ON cells
can hyperpolarize through modulation of their bipolar excitatory inputs. Direct inhibition
also offers another explanation as to why OFF cells are more rectified in their membrane
response than ON cells — directly hyperpolarizing responses in OFF cells demands a very
large conductance change. Because the conductance change most likely saturates, direct
inhibition can only hyperpolarize OFF ganglion cells to a certain point. Finally, ON cells
tend to have a lower spike threshold than OFF cells, as demonstrated by the SNL curves,
and this could explain why ON cells have a higher baseline spike rate (∼15 sp/s) than OFF
cells (∼5 sp/s). OFF cells need a larger membrane depolarization to produce the same spike
output.
3.3
Summary
By using a white noise stimulus, we can describe retinal processing with a simple model
composed of a linear filter followed by a static nonlinearity. Cross-correlating the ganglion
cell output with the white noise input allows us to determine the parameters of that model
60
that best describe computations performed by the retina. We found that the best model
for this processing consists of a biphasic impulse response that describes the temporal
structure of ganglion cell response and a rectified static nonlinearity that describes both
the ganglion cell’s synaptic inputs (membrane SNL) and spike generating mechanisms (spike
SNL). The simple model accounts for most of the ganglion cell response, and so exploring the
parameters of that model allows us to understand how the retina changes its computations
across different cell types and how it adjusts its computations under different stimulus
conditions.
ON and OFF cells are similar in their temporal structure, as demonstrated by their
identical impulse responses, yet differ in their nonlinearities. OFF cell inputs are more
rectified than ON cells and OFF cell outputs exhibit a higher threshold than ON cells. These
differences can be accounted for by discrepancies in spontaneous release rates at the bipolar
terminal and in the baseline spike rates, respectively. Such differences may be an artefact of
biological constraints — from the first synapse, ON and OFF pathways differ in how signals
are conveyed[99] and this difference may affect how signals are propagated to later synapses.
In addition, earlier studies[18] and our own observations have revealed that ON cells tend
to have larger receptive fields than OFF cells. Further exploration is needed to determine
why the differences between these two complementary pathways occur.
61
Chapter 4
Information Theory
The retina converts incident light into spike trains that it communicates to higher (cortical)
processing centers. The retina communicates these spikes through the optic nerve, which
presents a bottleneck through which the retina must efficiently send important information
about the visual scene. Because of metabolic constraints in this bottleneck, the retina
must encode this information using a limited number of spikes[65]. Clearly, attaching a
unique spike code to all aspects of a visual scene, whereby output activity directly mirrors
input signals, demands a high metabolic cost. For example, static scenes would generate
a persistent spike output and would therefore waste much of this energy on repetitive
information. Short, dynamic events would produce few spikes which would be lost in the
sea of static background activity.
To robustly encode changes in the visual scene while efficiently encoding background
static images, the retina performs computations on the input light signals so as to remove
redundancy and reject noise. To get at the computations performed by retinal preprocess-
62
ing, vision researchers have made quantitave predictions for how the retina encodes visual
information. Barlow observed that the first stages of visual processing reduce redundancy,
using few spikes to encode the most repetitive signals[5]. However, reducing redundancy
is not effective in transmitting information when the stimulus is noisy, as this redundancy
enables noise to be averaged out. A more analytical approach to deriving the spatiotemporal filters the retina uses to preprocess visual information is provided by information
theory. A number of researchers have used this approach to predict the optimal retinal
filters[2, 104, 103]. In this section, we adopt the information-theoretic approach and derive
the optimal spatiotemporal filter for the retina. We extend previous analysis to two dimensions, space and time. In addition, we make predictions as to how this filter changes as the
inputs to the retina change and verify these predictions with physiological results.
4.1
Optimal Filtering
The amount of information transmitted by a communication channel is defined as how much
the channel’s output reduces uncertainty about its input[92]. A communication channel
whose output is completely uncorrelated with its input transmits no information, as we
could never be able to deduce which input produced the observed output. However, a
channel that can consistently assign a unique output to every input transmits a lot of
information since we can, with confidence, determine every input signal for every output
signal we observe. The mutual information between input and output is therefore defined
by this reduction in uncertainty — the uncertainty of the input minus the uncertainty of
the input given the output we observe.
We can compute the uncertainty of a signal by calculating its entropy, which gives a
quantitative measure, in bits, of how many different possibilities the signal can represent.
63
For example, the entropy of a simple coin flip experiment is one bit, which represents two
possible outcomes (21 ). In the special case where the channel simply adds noise, the mutual
information is defined as the difference between the input’s entropy and the noise entropy.
When the channel noise is zero, the channel’s output reflect’s its input exactly, and therefore
the output entropy represents the same number of possible choices as found in the input. To
maximize information transmission through the channel, we should maximize the entropy
of the input and minimize the entropy of the noise.
Given an average power constraint, the ensemble with maximum entropy is a Gaussian.
With variance σ 2 , its entropy is log2 (2πeσ 2 )/2. If the noise in the channel is also Gaussian,
then the information rate, R, through the channel is defined as:
R = log2 (S(f ) + N (f )) − log2 (N (f ))
Z
S(f )
df
=
log2 1 +
N (f )
(4.1)
(4.2)
where S(f) and N(f) represent the power spectral density of the signal and noise, respectively, and where the integral is taken over all frequencies[92]. From Equation 4.2, we find
that the information rate is only logarithmically related to signal-to-noise ratio (SNR), and
so frequencies with very large differences in their SNRs contain similar amounts of information. Furthermore, because we integrate over all frequencies, information rate is linearly
proportional to bandwidth where signal power exceeds noise power. Hence, to attain high
information rates through the channel, our filter should transmit as many frequencies where
SNR> 1 as it can.
Transmitting signals, whether through the optic nerve or through another communication channel, is costly, and so we should encode them optimally. We can use total power,
64
Figure 4.1: Optimal Retinal Filter Design
A filter, F1 (f ), approximates the retina’s processing of visual scenes. Gaussian white noise, N0 , is added
to the power spectrum of visual input, S0 (f ). The retina filters these signals and produces an output.
Gaussian white noise, N1 , is added to the output producing a signal, S1 (f ), which is communicated
through the optic nerve.
P, of signals relayed through the channel as a measure of cost:
Z
P =
S(f ) + N (f )df
(4.3)
where S(f) and N(f) again represent the power spectra of the signal and noise respectively.
From Equation 4.2, we also find that we get less than one bit per dT (=1/df) in frequency
channels (df) where noise power exceeds signal power. However, from Equation 4.3, transmitting these noisy frequencies is still costly. Hence, our filter should reject these bands
where SNR< 1 so as not to waste channel capacity. In addition, our filter should also attenuate frequencies with SNR 1 because they carry only logarithmically more information
but use linearly more power.
Given these two strategies, we can begin to formulate our design for an optimal retinal
filter similar to the approach used by van Hateren[104]. For simplicity, we use the scheme
shown in Figure 4.1 which approximates all the computations that take place within the
retina as one filter with gain F1 (f ). The retinal filter receives an input signal, S0 (f ), and
outputs S1 (f ). Noise signals N0 and N1 are added to the signals at both filter input and
65
output, respectively. Using Equation 4.2, the information rate, I, is:
Z
I=
F1 (f )S0 (f )
df
F1 (f )N0 + N1
log2 1 +
(4.4)
and the power needed to transmit these signals, P, is:
Z
P =
F1 (f )(S0 (f ) + N0 ) + N1 df
(4.5)
where the integrals are taken over all frequencies. To find the optimal filter for the system,
we maximize the functional
Z
E[F1 (f )] =
log2
F1 (f )S0 (f )
1+
df − λ
F1 (f )N0 + N1
Z
F1 (f )(S0 (f ) + N0 ) + N1 df
(4.6)
where the first term represents information rate and where the second term represents how
much power it takes to transmit these signals. The multiplier, λ, is in units of rate/power
and therefore the cost factor 1/λ sets how much energy we are willing to spend to transmit
one bit of information.
Using calculus of variations, we can maximize the functional by setting the derivative
equal to zero and solving for F1 (f ):
(s
F1 =
4λ−1 (ln2)−1 /N1
2N0
1+
− 1+
S0 /N0
S0
)
S0
N1
S0 + N0 2N0
(4.7)
where we express S0 (f ), S1 (f ) and F1 (f ) as S0 , S1 , and F1 , respectively, for simplicity. To
66
find the power spectral gain of the retinal filter F1 , we must specify the power spectrum of
the input, S0 , and the noise, N0 and N1 . The retina is optimized to capture information
about natural scenes, which have a power spectrum proportional to 1/f 2 , as shown in
Figure 4.2. We assume noise in the system, N0 and N1 , is white, and therefore has a flat
power spectrum over all frequencies. For simplicity, we derive our optimal retinal filter in
one dimension of frequency without loss of generality.
To ensure that the filter gain remains positive, we must satisfy the inequality:
s
1+
4λ−1 (ln2)−1 /N1
S0 /N0
4λ−1 (ln2)−1 /N1
S0 /N0
1
λln2
S0
N0
≥ (1 +
>
∼
>
∼
2N0
)
S0
4N0 2
S0 2
N0 N1
S0
>
∼ N1 λln2
This sets a lower limit on the SNR. The lower the output noise power, N1 , relative to the
energy/bit cost 1/λ, the lower the SNR limit.
The above inequality determines the SNR where our filter, F1 , must cut off to preserve
positive information rates. Thus, how SNR depends on frequency determines where our
filter cuts off. Our optimal filter should reject all frequencies where SNR falls below λln2N1 .
We can now also determine the behavior of the filter, F1 , for frequencies where SNR 1.
Because S0 drops as 1/f 2 , this corresponds to low frequencies. In this domain, the square
root term of Equation 4.7 is close to 1 (since SNR 1), and we can use the linear expansion
for finding the square root. Hence,
67
Figure 4.2: Optimal Filtering
Natural scenes have a power spectrum S0 (f ) that is proportional to 1/f 2 and that intersects the power
spectrum of added Gaussian white noise at a given frequency, f0 . Maximizing information rates for a
fixed power constraint demands an optimal filter that peaks at this frequency, f0 . For frequencies less
than f0 , the filter’s behavior is proportional to 1/S0 (f ), whitening these frequencies at the output. For
frequencies greater than f0 , the filter cuts off, attenuating regions where SNR< 1.
(
F1 ≈
≈
≈
1 4λ−1 (ln2)−1 /N1 2N0
−
2
S0 /N0
S0
)
N1
S0
(S0 + N0 ) 2N0
1
( λln2
)
N1
−
S0
S0 + N 0
1
)
( λln2
S0
assuming N1 1/λln2. Thus, F1 ∝ S1 ∝ f 2 for low frequencies if the output noise N1 is
0
low. In this region, the filter acts to whiten the input, flattening it for all frequencies where
S0 > N0 . This makes sense since, from Equation 4.2, information rates depend linearly
on frequency where SNR> 1 and only logarithmically on SNR. Thus, for a fixed power
constraint, we should whiten the signal in the region where SNR> 1 to pass as many of
these frequencies as we can. From Figure 4.2, we see that in the region where SNR> 1, the
optimal filter flattens the power spectrum in the output signal.
68
We can extend the optimal filtering strategy derived above to two dimensions to gain
some intuition for how the retina optimally spatiotemporally filters visual signals. To fully
understand filtering properties in two dimensions, spatial frequency ρ and temporal frequency ω, we must also introduce a third parameter, velocity v. The power spectrum
expected for natural scenes is composed of images with a 1/ρ2 spectral distribution moving
with a distribution of velocities. Velocity is given by the ratio between temporal frequency
and spatial frequency, or v = ω/ρ. If an entire scene moves at a given velocity v, then every
spatial frequency ρ contained in that scene will translate to temporal frequencies ω = vρ
— thus, the entire spectrum lies along this velocity line. As the velocity of a scene changes
(think of changing the speed by which you turn your head, for example), every object within
that scene, which has an associated spatial frequency, will experience a change in temporal
frequency.
Thus, the spatiotemporal power is given by:
Z
R(ρ, ω) = Rs (ρ)
δ(ω − vρ)P (v)dv = Rs (ρ)P (ω/ρ)
(4.8)
where Rs (ρ) represents the spatial power spectrum of a static scene, which is well approximated by K/ρ2 [32] where K is constant and proportional to the power of the input signal.
δ(ω −vρ) is the Dirac delta function which is normalized to one; it is zero everywhere except
for ω = vρ. Hence, for a given velocity, v, we have a restricted set of ω and ρ that satisfy
this relationship. P (v) represents the probability of finding a certain velocity v in natural
scenes, and is given by
P (v) ∼
1
(v + v0 )n
69
(4.9)
Figure 4.3: Power Spectrum for Natural Scenes as a Function of Velocity Probability Distribution
Natural scenes are composed of images with identical Rn = 1/ρ2 power spectrums moving with a
distribution of velocities. The probability of observing each velocity decreases as 1/v 2 for velocities
greater than v0 .
where v0 and n are constant[32]. Intuitively, v0 represents a velocity threshold — velocities
smaller than v0 have a flat probability distribution while velocities larger than v0 have
probabilities that decrease as velocity increases. When observing a scene from a distance
of 10 m, v0 is ∼ 2 deg/s and n > 2[32].
From Equations 4.8 and 4.9, we see that each velocity has an identical spatial frequency
power distribution that is proportional to 1/ρ2 . However, the probability of observing
corresponding temporal distributions decreases as we increase velocity. Hence, the input
spectrum for natural scenes actually resembles the spectrum schematically shown in Figure 4.3, and is described by:
R(ρ, ω) =
1
K
K
=
ω
2
2
ρ ( ρ + v0 )
(ω + v0 ρ)2
70
(4.10)
where we set n = 2 in Equation 4.10[32] to simplify our analysis. Adding Gaussian white
noise to this power spectrum yields the input power spectrum shown in Figure 4.4a. For
low velocities, the contour where signal power intersects noise power for such a distribution
lies on a fixed spatial frequency, which we call ρ̂ in Figure 4.3. For high velocities, the
probability decreases, and so the contour where signal power intersects noise power lies on
a fixed temporal frequency.
Using the information theoretic approach, we can quantify these points and derive the
optimal spatiotemporal filter for the retina across all velocities:
F (ρ, ω) =
1
λln2
R(ρ, ω)
= K0 (ω + v0 ρ)2
(4.11)
The filter’s gain rises with both spatial and temporal frequency. More importantly, the
filter’s gain rises at velocities higher than v0 (i.e. ω/ρ > v0 ) to compensate for the decrease
in probability as velocity increases, and to therefore flatten this probability distribution.
To find where this filter cuts off, which also defines where the filter peaks, we revert to the
inequality derived above. We wish to only pass those temporal and spatial frequencies that
satisfy
R(ρ, ω)
N0
K
1
N0 (ω + v0 ρ)2
>
∼ N1 λln2
>
∼ N1 λln2
<
(ω + v0 ρ)2 ∼
K
N0 N1 λln2
For velocities less than v0 (i.e. ω/ρ < v0 ), the left side of the inequality is dominated by
71
Figure 4.4: Optimal Filtering in Two Dimensions
a) Natural scenes’ power spectrum R(ρ, ω) is approximated by Equation 4.10 and intersects the noise floor
along an “L”-shaped contour. b) Maximizing information rates for a fixed power constraint demands an
optimal filter that peaks along this contour. c) For velocities less than v0 , the filter’s peak is determined
by ρ̂. For velocities greater than v0 , the filter’s peak is determined by ω̂.
v0 ρ, and the filter peaks at a spatial frequency ρ̂ given as
1
ρ̂ =
v0
s
K
N0 N1 λln2
(4.12)
For velocities greater than v0 (i.e. ω/ρ > v0 ), the inequality becomes independent of v0 ,
and the filter peaks at a temporal frequency ω̂ given by
s
ω̂ =
K
N0 N1 λln2
(4.13)
Hence, for low spatial frequencies, the filter peaks at a fixed temporal frequency, ω̂, and for
low temporal frequencies, the filter peaks at a fixed spatial frequency, ρ̂. A three dimensional
representation of the optimal filter for natural signals described by Equation 4.10 is shown
in Figure 4.4b. The filter rises in both spatial and temporal frequency, to whiten the input,
and cuts off at a temporal and spatial frequency defined by ω̂ and ρ̂ above. In a two
72
dimensional ω-ρ plane, this peak defines an “L”-shaped contour, as shown in Figure 4.4c.
These temporal and spatial cutoffs define the peak of the optimal filter if we consider the
entire ensemble of stimulus velocities and optimize across the whole distribution. Intuitively,
the filter’s peak contour makes sense if we examine the spatial power spectrum along each
velocity line. At low velocities, which have a relatively high probability of occuring, the
spatial power spectrum goes as 1/ρ2 and intersects the noise floor at ρ̂. All velocities below
v0 have an equal probability of occuring, and thus the power in these signals is unchanged.
However, as we increase velocity above v0 , the probability begins to decrease. We can
interpret this reduction in probability as an effective reduction in the power of distributions
along these higher velocities. Thus, as we decrease that power, we expect the intersection
with the noise floor to lie at lower and lower spatial frequencies. From Figure 4.4, we see
that in this case, we indeed drop the spatial frequency at which our filter should peak, as
we move along the curve defined by ω̂.
The filter thus derived represents the retina’s optimal solution for efficiently encoding
an entire ensemble of distributions. To explore whether the retina actually realizes such
filtering, we turn to earlier studies. Psychophysical data has demonstrated that contrast
thresholds depend on an interplay between both spatial and temporal frequency, as shown
in Figure 4.5a[55]. For low temporal frequencies, the contrast threshold is relatively independent of spatial frequency. Similarly, for low spatial frequencies, the contrast threshold
is independent of temporal frequency. Peak sensitivity therefore also takes on an “L” shape
with its corner point at ρ̂ ∼ 3 cyc deg−1 and ω̂ ∼ 7 Hz. Velocity lines are included in the upper right of the figure to relate peak sensitivities to different velocities. These velocity lines
intersect the contour plots from the upper right to the lower left. If an entire scene moves
at a given velocity, we can then extract which spatial frequencies will evoke the strongest
response for that velocity and translate those spatial frequencies to their corresponding
73
Figure 4.5: Contrast Sensitivity and Outer Retina Filtering
(a) Contour plot of spatiotemporal contrast thresholds. The heavy line (max) represents peak sensitivity.
Sensitivities double from one contour to the next. Velocity is represented by the axis on the upper
right. The surface is symmetric around v=2 deg/s. Reproduced from [55] (b) Three-dimensional plot of
the magnitude of cone response for a purely linear circuit model of the outer retina. At higher spatial
frequencies, the bandpass temporal response becomes lowpass, and vice versa. Reproduced from [10]
temporal frequencies.
The contour plot of Figure 4.5a is remarkably similar to the three-dimensional plot
representing the optimal retinal filter for natural scenes with a power spectrum determined
by Equation 4.10. Both curves have peak contours that are “L”-shaped and defined by a
peak spatial frequency ρ̂ and a peak temporal frequency ω̂. Furthermore, the velocity line
that runs through the corner of the peak contour in both cases is ∼ 2 deg/s. This suggests
that the retina’s filter is indeed optimized for natural scenes. More remarkable, however, is
the fact that such a filter can be constructed with simple linear structures. The contour plot
of Figure 4.5a is similar to the three-dimensional plot generated by a purely linear model
of the outer plexiform layer (OPL) of the retina, shown in Figure 4.5b[10]. The linear
74
model is comprised of a network of electrically coupled cone cells that excite a network of
electrically coupled horizontal cells, which provide feedback inhibition back on to the cones.
The outer retina’s transfer function is bandpass in spatial frequency and remains fixed at the
same peak spatial frequency for low temporal frequencies. The bandpass spatial response
becomes lowpass as we move to higher temporal frequencies. Similarly, the transfer function
is bandpass in temporal frequency and remains fixed at the same temporal frequency for
low spatial frequencies. The bandpass temporal response becomes lowpass as we move to
higher spatial frequencies. Thus, the three-dimensional plot is also symmetric about a given
velocity line.
The peak of the outer retina’s transfer function, the peak of the psychophysical contrast
sensitivities, and the peak of the optimal retinal filter are all “L”-shaped. This suggests
that the outer retina’s filter is optimized for the entire ensemble of signals found in natural
scenes. However, although such filtering is ideal if we wish to capture all input velocities,
averaging over the entire distribution is suboptimal in the case where we stimulate the
retina with only one velocity. Ideally, we should determine how the static filter described
by Equation 4.11 affects the input spectrum along one velocity line, and then determine
how we should change this filter to maximize information rate for that specific velocity.
This implies that we need a second stage of filtering, potentially the inner retina, designed
to take outputs from the outer retina and modify them to optimally encode information by
dynamically adapting to that particular velocity line.
4.2
Dynamic Filtering
Retinal processing is designed to optimize information rates. However, because the optimal
filter depends on the power spectrum of the input, as in Equation 4.7, we expect that the
75
retina’s filter to change as the input spectrum changes. By dynamically adjusting its filter
to the spectrum presented, instead of averaging over the ensemble, the retina can optimally
encode signals over a large range of stimulus conditions. To gain a better appreciation
of how the retina might want to adjust its filters, let us examine the filtering strategy in
one dimension more closely. Assuming we stimulate the retina with the same 1/f 2 power
spectrum found in natural scenes, we expect the retina to optimize its filter such that
the peak of the filter lies where the signal power intersects the noise floor, as shown in
Figure 4.6a. In the figure, the initial power spectrum is represented by the solid blue line
whereas the retina’s filter is represented by the dashed blue line. Such filtering produces
the output power spectrum shown in blue in Figure 4.6b.
However, if the input power spectrum changes such that it intersects the noise floor at
a lower frequency (Figure 4.6a), then the output of the original retinal filter in Figure 4.6a
produces the output shown in Figure 4.6b. Clearly, such static filtering is sub-optimal.
While this filter whitens frequencies where SNR> 1, the filter also passes noisy regions
where SNR< 1. In fact, because the peak of the original filter now lies to the right of
the point where signal power intersects noise floor, the filter actually amplifies some of the
frequencies that are dominated by noise.
A better strategy would be for the retina to dynamically adapt its filter such that the
peak of the filter changes to the point where the new signal power spectrum intersects the
noise floor, as in Figure 4.6c. For the new power spectrum, the dynamic filter whitens
frequencies where SNR> 1 and attenuates frequencies where SNR< 1, as predicted by an
optimal filtering strategy. This generates an output power spectrum that is flat for SNR> 1
and attenuated for SNR< 1, as shown in Figure 4.6d.
In two dimensions, we can derive how the retina should adapt its filter by first, for an
76
Figure 4.6: Dynamic Filtering in One Dimension
(a) An optimal filter designed for the input power spectrum shown in blue peaks where signal power
intersects noise floor. b) If the input power spectrum changes such that SNR= 1 point lies at a lower
frequency, as shown by the red line, the filter is suboptimal. The original filter produces a whitened
response for low frequencies and a lowpass response for higher frequencies. However, the same static
filter now amplifies noisy regions where SNR< 1. (c) A dynamic retinal filter adjusts its properties such
that the peak of the filter lies where the new signal power intersects the noise floor, producing the
optimized filter output shown in (d).
77
image moving with speed v1 , deriving the output of the static optimal filter of Equation 4.11.
In this case, all spatial and temporal frequencies, ρ1 and ω1 , lie on the line ω1 = v1 ρ1 . The
power spectrum of signals along this velocity line, from Equation 4.8, is simply K/ρ21 ; since
we explicitly choose this velocity v1 , the probability of observing this velocity now becomes
1. The output of the static optimal filter, with this input, becomes
K
+ N0 K0 (ω1 + v0 ρ1 )2
ρ21
ω1
= KK0 ( + v0 )2 + N0 K0 (ω1 + v0 ρ1 )2
ρ1
R(ρ1 , ω1 )F (ρ1 , ω1 ) =
(4.14)
(4.15)
where the first term represents the signal power and the second term represents the noise
passed through the outer retina’s filter. Since we are only stimulating with the velocity
curve v1 , we are only concerned with temporal and spatial frequencies ω1 and ρ1 that lie on
this curve. The power spectrum of the output of the outer retina’s static filter is represented
in Figure 4.7a. The output power spectrum is flat for velocities v1 less than v0 , but rises
as the square of velocity for velocities greater than v0 . This makes sense intuitively since
input spectrums with low velocities are dominated by a 1/ρ2 spatial power spectrum which
is whitened by the outer retina filter. At higher velocities, the outer retina is designed
to compensate for the drop in probability of seeing high velocities and therefore amplifies
signals with these velocities.
To determine how the inner retina should compensate for this distribution, we revert to
our optimal filtering strategy. We know that to whiten this signal, our inner retina filter
should be the inverse of the signal power in the outer retina output. Hence, the inner
retina’s filter has a profile described by
78
Figure 4.7: Inner Retina Optimal Filtering in Two Dimensions
a) The output of the outer retina’s optimal filter F (ρ, ω), approximated by Equation 4.15, is flat for
velocities less than v0 and rises with velocity for velocities greater than v0 . b) The optimal inner retina
filter is designed to compensate for outer retina filtering by whitening the power spectrum for all velocities.
For velocities less than v0 , the filter is flat since the outer retina has already whitened the input. For
velocities greater than v0 , the filter’s gain drops with velocity to compensate for the outer retina’s
amplification of these velocities (projected on to the velocity axis in the upper right).
FIP L (ρ1 , ω1 ) =
=
1
λln2
ROP L (ρ1 , ω1 )
1
λln2
KK0 ( ωρ11
= K1
( ωρ11
1
+ v0 )2
1
+ v0 )2
(4.16)
(4.17)
(4.18)
where FIP L (ρ1 , ω1 ) represents the filter we need to implement in the inner retina to maintain
optimal signaling, and where ROP L (ρ1 , ω1 ) represents the signal power of the output of
our static outer retina filter. A three dimensional representation of the inner retina filter
described by Equation 4.18 is shown in Figure 4.7b.
In the case where v1 < v0 , the outer retinal filter’s output is flat since Equation 4.15
simplifies to KK0 v02 . The outer retina has done its job in whitening input signals, and so
79
the inner retina’s filter should remain flat in this region and maintain the same cutoffs. For
spatial frequencies, this cutoff, ρˆ1 , corresponds to the same ρ̂ we found in Equation 4.12.
The peak spatial frequency is independent of velocity in this domain and remains fixed at
ρ̂. Similarly, the temporal frequency at which the inner retina should cutoff, ωˆ1 , should also
correspond to the same ω̂ we found in Equation 4.13. Thus, we find
v
ωˆ1 = v ρˆ1 =
v0
s
K
N0 N1 λln2
(4.19)
The temporal frequency at which the inner retina filter should cut off, ωˆ1 , increases linearly
with velocity. The intersection between the individual spatial power spectrum and the
noise floor determines ρ̂ = ρˆ1 , and therefore determines where the optimal filter cuts off in
space. Intuitively, if the stimulus ensemble consists of the same 1/ρ2 images moving with
a distribution of velocities, then there is nothing left to do after spatial filtering whitens
the spectrum in this region, since the temporal frequencies produced by motion will also be
white.
In the case where v1 > v0 , the outer retinal filter’s output, from Equation 4.15, is
described by KK0 (ω1 /ρ1 )2 . Clearly, the magnitude of the signal increases with velocity,
reflecting the gain we see in the static outer retina filter. Therefore, in this region, the
inner retina filter has a gain that decreases with stimulus velocity, matching the probability
distribution of velocities in natural scenes. This makes intuitive sense since we need to
compensate for the gain in the outer retina filter by the inverse of the outer retina’s velocity
dependence. To determine where the inner retina filter cuts off, and therefore peaks, for
velocities greater than v0 , we must first determine how the input noise, N0 , is filtered by
the outer retina. Since both signal, S0 , and noise, N0 , along a given velocity line that has
probability equal to one are filtered in the same way by the outer retina, their ratio should
80
be unchanged. Hence, our inner retina filter should peak at the spatial frequency ρˆ1 that is
identical to the peak spatial frequency we found in our outer retina analysis:
ρˆ1 =
K
N0 N1 λln2
1/2
The inner retina should cut off at the same fixed point in spatial frequency, and after
translating through velocity to temporal frequency, should cut off at a temporal frequency
that increases linearly with velocity. For these higher velocities, however, the outer retina
filter has already attenuated temporal frequencies greater than ω̂ and so although the inner
retina would maintain a cutoff at ρˆ1 if we ignore outer retina cutoffs, passing temporal
signals larger than ω̂ is unnecessary in the inner retina since these frequencies in the outer
retina’s output are already attenuated. Thus, the inner retina should just maintain the
cutoff at ω̂ for velocities greater than v0 .
Because the inner retina compensates for outer retina filtering and adjusts its cutoffs
accordingly, the inner retina represents a dynamic stage in our optimal filtering strategy.
The adaptation realized by the inner retina is in response to velocity — for low velocities,
the inner retina senses the velocity passed through the outer retina and sets its cutoff
to maintain the same optimal filtering strategy dictated by the outer retina. For higher
velocities, the inner retina maintains the same temporal frequency cutoff we found in the
outer retina, ω̂, but compensates for outer retina filtering by attenuating signals with higher
velocities.
In addition to adapting to velocity, however, the retina’s optimal filter should also adapt
to different levels of stimulus contrast since contrast is not constant across all image conditions. These changes in input contrast correspond directly to changes in stimulus power
81
and so when we increase contrast, we effectively increase the constant K in Equation 4.10.
Stimulus power determines where our optimal filter should cut off, and so changing this
power demands adaptive changes in these temporal and spatial cutoffs to maintain optimal
signaling. For low velocities, the spatial cutoff, ρ̂, is determined by the outer retina and is
given in Equation 4.12. This spatial cutoff translates to a temporal cutoff which the inner
retain maintains, attenuating temporal frequencies greater than ωˆ1 , and which depends on
velocity v1 and is given in Equation 4.19. We can see that both of these equations depend
on stimulus power, K. Increasing contrast will increase both the spatial and temporal
frequency at which the optimal filter should cutoff and so we expect increasing stimulus
contrast will adjust the retina’s optimal filter such that it passes higher frequencies. For
high velocities, we found that optimal filtering is also dominated by the outer retina, which
sets a temporal frequency cutoff, ω̂, determined by Equation 4.13. Because temporal frequencies larger than ω̂ are attenuated before reaching the inner retina, we expect that the
inner retina maintains this same temporal frequency cutoff. This cutoff, ω̂ also depends on
stimulus power, as we can see in the equation. Increasing the stimulus contrast increases
K and pushes this cutoff out to higher temporal frequencies.
The ability of the retina to adapt its temporal frequency profile in response to different
stimulus contrasts is one of the hallmark’s of the contrast gain control mechanism, first described by Victor and Shapley[93]. This nonlinearity in retinal processing makes the retina’s
filter faster and less sensitive as stimulus contrast increases. From our information theoretic
analysis, we can see the speed up is consistent with an optimal filtering strategy, since such
an optimal filter would pass higher frequencies as input contrast increases. Furthermore,
we know from our analysis above that the optimal filter is related to the inverse of input
stimulus. Hence, increasing stimulus power directly decreases the retinal filter’s gain. This
suggests that the contrast gain control mechanism is not simply an artefact of biological
constraints, but that it is consistent with a strategy aimed at efficiently encoding visual
82
information.
We have thus derived the optimal retinal filter for the ensemble of signals found in natural
scenes that adjusts to both stimulus velocity and stimulus contrast. A three dimensional
plot of the optimal retinal filter, generated by combining the outer and inner retina’s optimal
filters, is shown in Figure 4.8. The filter is simply bandpass in space and peaks at the spatial
frequency, ρ̂, derived above. Out hypothesis is that the outer retina provides a static filter
that is optimized to the average of the entire ensemble — it has a spatiotemporal profile
that is inversely related to the velocity probability distribution — while the inner retina
adapts to individual velocities. The extent of inner retina filtering is determined by the
velocity of the input signal. Distributions that lie on velocities less than v0 are simply cut
off at the fixed spatial frequency, ρ̂, while distributions that lie on velocities greater than
v0 must be attenuated in the inner retina to maintain a whitened output. In the first case,
the filter peaks along the velocity line at a fixed spatial frequency ρ̂ and at a temporal
frequency that increases linearly with velocity. In the second case, the inner retina whitens
outer retina outputs and maintains the same temporal frequency cutoff at ω̂. In both cases,
the input stimulus has a power spectrum that is ∝ 1/ρ2 . The retina’s optimal filter, after
combining the outer and inner retina, ignores probabilities of velocities and simply whitens
these signals by implementing a bandpass spatial filter. To verify that the retina indeed
sets its spatiotemporal peak at this point, we turn to physiology.
4.3
Physiological Results
Using the same recording methods as in our white noise analysis, we recorded the intracellular responses of guinea pig ganglion cells to visual stimuli of different velocities (Figure 4.9a).
We presented the ganglion cell with a drifting grating whose luminance varied sinusoidally
83
Figure 4.8: Retinal Filter
Combining the filters derived for the outer and inner retina yields an optimal filter that is bandpass in
space, whitening input stimuli of different velocities that have the same 1/ρ2 power spectrum. The
outer retina’s filter takes the statistical distribution of velocities into account, while the inner retina
compensates for this averaging and produces a whitened signal at its output.
in the horizontal direction but was constant in the vertical direction. We varied the velocity
of the grating and computed the amplitude of the first Fourier component at each temporal frequency. By measuring how the temporal and spatial profiles change with different
velocities, we can understand how the retina is optimized to change its filter with different
input velocities.
As we change the velocity of the drifting grating and measure the temporal frequency
profile, we find that the peak temporal frequency increases with velocity (Figure 4.9b, n=4).
However, we find that the peak spatial frequency remains unchanged as we increase velocity
(Figure 4.9c, n=4). The peak temporal frequency is linearly related to velocity while the
peak spatial frequency is fixed for all velocities we used to stimulate the ganglion cell
(Figure 4.9d). This suggests that the peak temporal frequency of the retina’s dynamically
changing temporal filter is governed by optimal filtering in the outer and inner retina. In
the above analysis, this implies that v ρ̂ determines where the optimal filter places the peak
of its temporal response. This data also suggests that the velocities we used to explore
84
Figure 4.9: Intracellular Responses to Different Velocities
(a) We record intracellular responses from guinea pig ganglion cells while presenting the retina with a
drifting sinusoidal grating. The grating’s luminance is constant in the vertical direction. By varying the
velocity of the grating, we can determine how peak temporal and spatial frequency responses change.
(b) Increasing velocity (v, in deg/s) causes a rightward shift in temporal frequency responses. All curves
from different stimulus velocities are overlayed in the bottom right panel. (c) Increasing velocity has no
effect on peak spatial frequency. All curves from different stimulus velocities are overlayed in the bottom
right panel. (d) Peak temporal frequency increases linearly with velocity while peak spatial frequency
remains constant
85
retinal filtering were not high enough to investigate the regime where v > v0 in the guinea
pig, since the data demonstrates that filtering remains fixed at a single spatial frequency.
For the low velocities that we did explore, the analytical expression for natural scenes
(Equation 4.8) states that the spatial cutoff is fixed at a spatial frequency ρ̂. Hence, the
behavior imposed by retinal filtering is consistent with an optimal filtering strategy if we
assume that ensembles of signals in natural scenes are probabilistically distributed — for
these low velocities, linear increases in velocity cause a linear increase in the peak temporal
frequency.
4.4
Summary
By taking advantage of information theoretic approaches, we can derive what the retina’s
optimal filter ought to be given a certain input power spectrum. To maximize information
rates, the optimal filter is one that whitens frequencies where signal power exceeds the noise,
peaking at a cutoff determined by stimulus and noise power, and that attenuates regions
where noise power exceeds signal power. The filter thereby realizes gains in information
rate by passing larger bandwidths of useful signal while minimizing wasted channel capacity
from noisy frequencies. In addition, we can also predict how this filter changes with changes
in the input spectrum. If we consider changes in input velocity, we find that the optimal
temporal filter moves its peak linearly with velocity.
From the psychophysical data and from the linear model for the outer plexiform layer, we
find that outer retina filtering is consistent with the optimal filtering strategy if we wish to
construct a static filter that averages over all input velocities. Remarkably, the outer retina
realizes this optimization with a fixed linear filtering scheme. The inner retina’s ability to
adjust its cutoff frequency may be important in further optimizing the retina’s filter when
86
we stimulate with a particular velocity and in helping the retina attenuate high frequencies
where noise power exceeds signal power. For low input velocities, inner retina filtering tracks
input velocity to maintain optimal filtering. Thus, inner retina filtering would have to be
adaptive so as to determine how its corner frequency changes with input stimulus velocities.
For high input velocities, inner retina filtering may act to whiten outputs from the outer
retina to maintain an optimal encoding strategy. Finally, because the inner retina moves its
corner frequency in response to input velocities, this adaptation may have implications for
more complex stimuli. Signals within the inner retina are communicated laterally through
amacrine cells. If the inner retina at a particular location adjusts its corner frequency in
response to an input velocity, the activity that reflects this adjustment may affect the inner
retina at other locations. For example, if large regions of the retina are stimulated with the
same velocity, the corner frequency set by this velocity in the inner retina may change the
response dynamics in other regions of the retina. Through this mechanism, we hypothesize
that the retina may be able to dynamically change its filtering scheme by averaging the
effect of velocity at different spatial locations.
87
Chapter 5
Central and Peripheral Adaptive
Circuits
In the previous chapter, information theoretic considerations led us to a mathematical
expression for the retina’s optimal filter. Dynamic filtering in the retina allows the retina to
adapt to different input stimuli and to maximize information rates for those stimuli. While
we predict that the retina changes its filters because stimulus velocities in natural scenes
demand different filtering strategies, we also wish to explore how adaptations are realized
in response to other elements found in natural scenes. We wish to quantify how the retina
adjusts its filters for different stimulus contrasts, and how the retina changes its response
to a specific stimulus when presented against a background of a much broader visual scene.
Furthermore, we would like to reach a description of the cellular mechanisms underlying
these adaptations and theorize why the retina chooses these mechanisms in particular.
White noise analysis gives us a powerful tool for exploring these questions. Through
88
the linear impulse response and static nonlinearity characterized using white noise analysis,
we can directly examine how retinal filters change with different stimulus conditions. To
simplify our analysis, we focus on the linear impulse response because it tells us how the
retina filters different temporal frequencies in the visual scene. Such an analysis can also
extend to spatial filtering by using spatial white noise, but we focus on temporal filtering
for simplicity. In this chapter, we examine the changes in the ganglion cell’s linear impulse
response as we increase stimulus contrast and compare those changes to those observed when
we introduce visual stimuli in the ganglion cell’s periphery. We propose a simplified model
for two parallel mechanisms that mediate adaptation of the retinal filter, one local and one
peripheral, and present preliminary data detailing the cellular interactions underlying these
mechanisms.
5.1
Local Contrast Gain Control
A purely linear representation of retinal filtering provides an attractive initial description of
how the retina processes information, as such a representation is easy to conceptualize. Linear systems exhibit the properties of superposition and proportionality, and hence knowing
a system’s linear impulse response allows one to predict the system’s output for any given
input through a trivial convolution. Rodieck made an early attempt at quantifying retinal
processing through the use of such a linear representation in describing ON ganglion cell
responses to a flash of light[83]. Rodieck’s model asserted that ganglion cell responses can
be predicted by summing these linear impulse responses, both in space and in time, through
a weighting function. Subsequent work demonstrated that weighted spatial summation of
linear responses does not hold for all ganglion cell responses — for example, surround signals are delayed in the center response[38, 88, 6], yet descriptions of retinal processing still
89
relied on purely linear filters in time[41].
The linear relationship between input and output detailed by these filters yielded very
good initial predictions for ganglion cell responses, but these predictions only held under
certain conditions. Specifically, for such a linear filter to capture most of the response
behavior, modulations in the input signal must be small relative to the mean[40]. Such
constraints are hardly representative of the ensemble of signals presented to the retina in
natural life. Hence, a more accurate description of retinal processing must include some
nonlinear behavior, whereby the retina dynamically adjusts its linear filter depending on
the input stimulus. One of these nonlinearities is “contrast gain control,” first described by
Victor and Shapley[93, 94], which causes a change in the properties of the retina’s linear
filter that depends on signal contrast. When stimulated with larger light fluctuations, the
retina’s response becomes less sensitive and faster. In a model capturing the properties of
this nonlinear behavior, Victor showed that such a change in ganglion cell response comes
from a contrast dependent speed up in the retinal filter’s time constant[107].
From an information theoretic standpoint, we can see how the adjustments realized by
contrast gain control make sense through some simple observations. In Section 4.1, we
derived the optimal retinal filter for capturing information contained in natural scenes and
found that such a filter has a behavior that depends on the power spectrum of natural scenes,
S0 (f ), a function of both spatial and temporal frequencies (Equation 4.8, here generalized
as f ). The optimal filter for such a spectrum is ∝ 1/S0 (f ) and cuts off where noise power
exceeds signal power (for details, see Section 4.1). Measurements have shown that natural
signals have a power law that is ∝ 1/f 2 . As we showed in Section 4.2, in the case where we
increase signal contrast, and thus increase signal power, we effectively increase the frequency
at which the signal power intersects the noise floor. Hence, we expect the optimal filter to
pass higher frequencies in the high contrast case, making the system faster. In addition,
90
Figure 5.1: Recording ganglion cell responses to low and high contrast white noise
We recorded the ganglion cell response to alternating ten second epochs of a white noise stimulus whose
depth of modulation switched between 10% and 30% contrast. We presented the white noise stimulus as
fluctuations in intensity of a 500µm spot centered on the ganglion cell’s receptive field (top). Recorded
responses are shown for three such epochs (low contrast, high contrast, low contrast). For each trace, we
extracted the membrane potential, shown in red, and the spikes to compute both membrane and spike
impulse responses.
because we are increasing the input signal power, and because the optimal filter is inversely
related to the input spectrum, we also expect the filter’s gain to decrease in the high contrast
case, making the system less sensitive.
To directly explore the contrast gain control mechanism in ganglion cell responses, to
investigate the retinal filter’s dependence on temporal contrast, and to elucidate some of the
mechanisms underlying this nonlinear behavior, we use our white noise analysis described
in Section 3.1. We recorded intracellular responses from guinea pig retinal ganglion cells
91
as we presented a low and high contrast white noise sequence, and measured both the
membrane and spike impulse response under these two conditions. We focus on the impulse
response because we wish to explore the nonlinear effects of stimulus contrast on the retina’s
temporal filter. The impulse response directly tells us how the ganglion cell responds to
different frequencies, and thus directly tells us how its frequency sensitivity changes under
different conditions. In addition, we restrict our analysis to OFF Y cells, since these cells
tend to exhibit a larger contrast effect[56, 18] and a larger effect from peripheral signals
(see below).
Our stimulus was a 500µm spot, centered over the ganglion cell’s receptive field, whose
intensity was governed by a white noise sequence, whose standard deviation, σ, relative
to its mean, µ, served as a measure of contrast, ct = σ/µ. We alternated between tensecond epochs of a 10% and 30% white noise sequence for four minutes and recorded the
ganglion cell response. A typical ganglion cell response to three of these epochs is shown in
Figure 5.1. As the stimulus modulation depth increased from low contrast to high contrast,
ganglion cell responses became larger, as expected. To quantify the change in response
with contrast, we cross-correlated the ganglion cell output with the white noise input for
each of these conditions. As described in Section 3.1, we normalized the impulse response
computed for each condition by that condition’s stimulus power so that we could compare
how the impulse response changes across conditions.
The membrane and spike impulse response computed by cross-correlating the output
with the input are shown in Figure 5.2. We normalized the curves to the peak of the low
contrast impulse response. From the figure, we find that as we increase modulation depth,
the ganglion cell’s impulse response decreases in magnitude, consistent with our prediction
that sensitivity decreases as we increase stimulus power. In addition, we noted a slight
speed up in the peak of the impulse response under high contrast conditions, suggesting a
92
speed up in the retinal filter.
The membrane and spike static nonlinearities are also shown in Figure 5.2 and we found
that as we increased stimulus contrast, the shape of the static nonlinearity changed. To focus
on how contrast affects the impulse response, and to simplify our analysis, we eliminated
any contrast-induced variation in the static nonlinearity. We found that we could make
the static nonlinearity in the two stimulus conditions contrast-invariant through a simple
scaling of the x-axis, an approach similar to that used by Kim[56] and Chichilnisky[18].
Because the white noise analysis provides a non-unique decomposition, we have the liberty
to scale either the impulse response or static nonlinearity, as long as we compensate for
the scaling in one with a scaling in the other, and maintain the same overall retinal filter.
The combination of the impulse response and the static nonlinearity determines the retina’s
overall temporal filter. Ordinarily, we would have to look at the output of these two stages
to compare responses across conditions. However, fixing one of these stages, the static
nonlinearity, allows us to consider changes in the impulse response as representative of all
the changes in the overall filter.
Membrane and spike static nonlinearities are fit with the cumulative distribution function described by Equation 3.11:
N (x) = αC(βx + γ)
A typical curve produced by this function is shown in Figure 5.3a. Scaling the x-axis
corresponds to multiplying the middle parameter, β, by a scaling factor, which we call ζ.
β determines the slope of the function that fits the static nonlinearity, and multiplying β
by ζ increases the slope when ζ > 1 and decreases the slope when ζ < 1. We found a value
93
Figure 5.2: Changes in membrane and spike impulse response and static nonlinearity with
modulation depth
For each stimulus condition (10% and 30% contrast), cross-correlation of the ganglion cell membrane
and spike response generates the membrane (top) and spike (bottom) impulse response. The linear
prediction of the impulse response, mapped to the recorded membrane and spike output, generates the
static nonlinearly curves shown on the right. Increasing stimulus contrast causes a change in static
nonlinearity. Impulse responses are normalized to the peak of the low contrast impulse response, and
static nonlinearities are normalized to the peak of the high contrast static nonlinearity.
94
of ζ for the low contrast static nonlinearity that produced an overlap of the low and high
contrast static nonlinearities, and divided the low contrast impulse response by this value
of ζ as in Figure 5.3a. Intuitively, such a transformation makes sense since expanding the
extent of the linear prediction (increasing the x-axis by multiplying by ζ < 1) in the static
nonlinearity corresponds to increasing the gain of the linear filter (dividing the impulse
response by ζ < 1).
The scaled spike static nonlinearity and impulse response for the curves in Figure 5.2
are shown in Figure 5.3b. We show the spike response to demonstrate the principle, but the
same procedure determines scaling of the membrane response. We normalized the impulse
responses shown by the peak value of the low contrast impulse response. In this case, to
scale the nonlinearity we multiplied the slope of the distribution function describing the
static nonlinearity by a value of ζ < 1, which corresponds to dividing the impulse response
of Figure 5.2 by ζ. Making the static nonlinearities contrast-invariant further increases the
gain reduction in spike impulse response as we go from low to high contrast. We recorded
the change in scaled membrane and spike impulse response between low and high contrast
stimulus conditions for 17 cells. The average response, normalized by the low contrast
peak, for both membrane and spike is shown in Figure 5.3c. Shaded regions represent
SEM and are colored according to the stimulus condition. Because the impulse responses
by themselves do not describe differences in the temporal filter until we scale the static
nonlinearities to make them contrast-invariant, we focus on the scaled impulse response to
draw conclusions about the low and high contrast conditions. As evidenced by the data,
increasing signal power from 10% to 30% contrast causes a consistent gain reduction in both
the membrane and spike impulse response, and a slight speed up in peak response. The
subtle timing change in the membrane response suggests that presynaptic circuits adapt
to the higher contrast levels. The timing change is more pronounced in the spike impulse
response, however, suggesting that cellular properties of the ganglion cell’s spike generating
95
Figure 5.3: Scaling the static nonlinearities to explore differences in impulse response
a) To make the static nonlinearity contrast-invariant, we scale the x-axis of static nonlinearity, thus
changing the slope of the nonlinearity. This change in slope can be compensated for by scaling the
impulse response amplitude (y-axis). b) The spike static nonlinearities of Figure 5.2, scaled so that the
two contrast conditions overlap. In this case, we reduced the slope of the low contrast static nonlinearity,
which translated to increasing the gain of the low contrast impulse response. c) The scaled membrane
and spike impulse responses computed for both 10% and 30% contrast stimuli for 17 OFF cells. Traces
represent average impulse response. Shaded regions represent SEM, and are colored dark gray for 10%
contrast and light gray for 30% contrast. Increasing stimulus contrast reduces the system’s gain and
causes a slight speed up which was more pronounced in the spike impulse response.
96
Figure 5.4: Root mean squared responses to high and low contrast stimulus conditions
For each cell, we averaged the root mean square membrane potential (left) and spike rate (right) across
all epochs of each stimulus condition. Increases in stimulus contrast cause an increase in RMS membrane
potential and spike rate that decays over time, while decreases in stimulus contrast cause a decrease in
RMS that gradually increases. Each cell’s RMS response was fit with a decaying exponential (gray). The
RMS membrane potential and spike rate averaged over all cells is shown on the bottom. RMS responses
are normalized by the peak RMS response, and we express membrane RMS as fluctuations around the
resting potential.
mechanism also depend on input power, consistent with earlier studies[56]. The larger
timing change in the spike response is probably of more consequence for visual processing
since it is the spike data that is relayed to higher cortical structures.
The effect of changing temporal contrast on ganglion cell response is not invariant with
time, but instead has a time course we could measure. Such contrast adaptation has been
recorded in earlier studies[56, 95] and has been described as a different mechanism than
Victor’s instantaneous contrast gain control. To explore this change in sensitivity with
time, we compute the root mean square (RMS) membrane potential and spike rate for each
ten second epoch and average across stimulus conditions. The RMS membrane potential,
with resting potential subtracted, is shown with the RMS spike rate in Figure 5.4. For
one cell, Figure 5.4 shows that at the onset of a high contrast stimulus, both membrane
97
potential and spike rate are initially larger, but decline over time as the cell’s sensitivity
decreases. We averaged the RMS responses across all the cells and found that this behavior
is consistent. We fit the time course of this decline with a decaying exponential whose time
constant is 2.34 ± 0.28 sec for membrane potential and 0.86 ± 0.24 sec for spike rate. The
longer time constant for membrane RMS is attributable to the fact that the initial change
in membrane RMS is small relative to the baseline membrane RMS, and so although there
is a decay with time, the decay is not very dramatic. When the stimulus contrast reverts
back to 10% contrast, the RMS responses are initially small but gradually increase as the
cell recovers its sensitivity. We also fit this time course with a decaying exponential for both
membrane potential (average of 4.26 ± 0.81 sec) and spike rate (average of 3.25 ± 0.48 sec).
Because the ganglion cell RMS response changes with time, we explored how the linear
kernel and static nonlinearity change with time as we alternated between low and high
contrast white noise stimuli to see if the slow adaptation affected the instantaneous gain
changes we observed. We divided each ten second epoch into five periods of two seconds
each, as shown in Figure 5.5a, and measured the linear kernel and static nonlinearity,
averaged across the entire experiment. Hence, the linear-nonlinear parameters we measured
in the first two seconds of the high contrast condition, for example, were averaged from the
first two seconds of every high contrast ten second epoch. We then set the last two second
period of the low contrast condition as the reference condition and scaled the membrane
static nonlinearities from the remaining periods to the membrane static nonlinearity from
this reference period. Because the white noise model presents a non-unique solution, scaling
the static nonlinearities allows us to directly compare how the system impulse response
changes with time. We chose the last two seconds of the low contrast condition to compare
across other experiments (see below) because by these last two seconds, the ganglion cell
response has reached steady state. As shown in Figure 5.5b, we allowed the remaining static
nonlinearities to change their slope and to have a vertical offset to match the reference static
98
nonlinearity. The change in slope translates to a gain change in the impulse response while
the vertical offset only reflects a tonic depolarization or hyperpolarization in the membrane
response. We compared the impulse responses of the remaining periods after rescaling them
with their associated change in static nonlinearity slope.
We concentrated on the change in membrane impulse response because we found that
the transformation from membrane response to spike response is independent of the experimental condition. To verify this, we recorded the membrane response and spike rate at
every time point in a typical experiment and mapped the relationship between membrane
voltage and spike rate. This algorithm is identical to the algorithm we used to determine
the static nonlinearity by mapping between linear prediction and ganglion cell response.
The membrane to spike mapping is shown for two cells in Figure 5.5c. The points represent
the average spike rate for each membrane voltage, and the error bars represent SEM. For
both cells, the curves for 10% and 30% contrast are identical, although the high contrast
curve spans a larger range. In general, as membrane voltage increases, spike rate increases
monotonically, independent of stimulus contrast. In the cases where spike rate does not
increase monotonically, as shown in the cell on the right, the relationship still remained
identical for low and high contrast. Thus, our analysis of the membrane impulse response
and static nonlinearity is sufficient to account for the entire ganglion cell response.
The peak values of the impulse responses measured in the five two second periods for
both low and high contrast conditions, scaled as described above and normalized to the impulse response measured in the last low contrast period, are shown in Figure 5.6a. In the low
contrast conditions, the peak values are consistent across the five periods and have a value
around 1. In the high contrast conditions, the peak values are also consistent across the
five periods, but have a value around 80% of the low contrast peak. This suggests that the
contrast gain control mechanism that changes the retina’s gain is instantaneous and persis-
99
Figure 5.5: Computing linear kernels and static nonlinearities for two second periods of
every epoch
a) We divided each ten second epoch of low and high contrast response into five two second periods. We
computed the linear kernel and static nonlinearity for each period and averaged corresponding periods
across the entire experiment. b) We set the membrane static nonlinearity of the last period in the low
contrast condition as the reference and scaled the membrane static nonlinearities from the remaining
periods to this reference. We allowed both the slope (black arrows) and the vertical offset (red arrows)
of the membrane static nonlinearities to change to match the reference static nonlinearity. The change in
slope directly changes the gain of the impulse response while the vertical offset indicates a tonic change
in mean membrane response and does not change the gain or timing of the impulse response. c) We
plotted the transformation from membrane voltage (mV) to spike rate (sp/s) for both low and high
contrast ganglion cell responses. This mapping was identical in the two conditions, although the high
contrast curve spanned a larger range, since the responses are larger. The mapping was identical in the
two conditions for both a cell with a monotonically increasing membrane-spike relationship (left) and a
cell with a non-monotonically increasing relationship (right). Circles represent the average spike rate for
each membrane potential and error bars represent SEM.
100
tent for the entire ten second epoch. Earlier studies had found that the gain of the ganglion
cell’s response changes slowly with time, a mechanism called contrast adaptation[95, 56].
However, this change in gain could be attributed to the non-uniqueness of the white noise
analysis. These studies computed the gain of the linear impulse response by cross-correlating
the ganglion cell response with the stimulus without adjusting for non-uniqueness by scaling
static nonlinearities. They found that the gain of the impulse response decreased with time
when stimulus contrast increased. However, one can only compare these impulse responses
if they are a unique representation of retinal filtering. In our data, we found that the gain
of the unscaled impulse response also decreased with time after we increased stimulus contrast, but after scaling the nonlinearities, this change in gain was eliminated. Our results
suggest that the contrast adaptation observed in these earlier studies may be an artefact
of the non-uniqueness of the white noise analysis. We scaled the static nonlinearities to
directly compare the impulse responses in low and high contrast and found that such a
scaling eliminates any temporal changes in the gain of the impulse response.
We also measured how the time-to-peak of the impulse response changes with time when
we switch between the low and high contrast conditions. The change in peak time, expresses
as a percentage change from the peak time of the last period in the low contrast condition,
is shown for all periods in Figure 5.6b. In the low contrast condition, the percentage change
for all periods is roughly zero, suggesting that the peak time remains consistent across time.
In the high contrast condition, however, the percentage change in peak time rises in the
first period, and only reaches steady state by the second period. This implies that while
the gain change is instantaneous, the timing change we observe in the impulse response
increases over time until reaching a steady state value.
We interpreted the change in vertical offset needed in fitting the static nonlinearities to
the static nonlinearity computed in the last period of the low contrast condition as a tonic
101
change in mean of the membrane response. This change has no effect on the gain or timing
of the impulse response. We expressed this vertical offset as a percentage of the range of
membrane responses during that particular period. This allows us to compare the change in
vertical offset across periods and across cells. The vertical offsets thus calculated are shown
in Figure 5.6c. We found that in both the low and high contrast conditions, the change
in vertical offset was unremarkable. Hence, as we qualitatively observed in Figure 5.4,
alternating between low and high contrast conditions does not change the mean of the
membrane response as much as it changes the range over which the membrane response
fluctuates.
Finally, we recorded the total number of spikes occuring within each two second period
to measure how mean spike rate changes across conditions, shown in Figure 5.6d. When the
white noise stimulus switched to 10% contrast, the mean spike rate dropped and remained
low across the entire ten seconds. When the stimulus switched to 30% contrast, the mean
spike rate immediately rose and then exhibited a very slight decrease over the five two second
periods, but the change (1-2 spikes) was not significant compared to the SEM. Hence, the
change in spike rate between low and high contrast was fixed across time, consistent with
the instantaneous and persistent change in impulse response.
The contrast adaptation behavior we observed in our RMS measurements are qualitatively similar to that observed in earlier studies, although our spike rate time constants
are shorter. This difference may be a result of the retina’s ability to adapt its sensitivity’s
temporal profile to different periods of contrast fluctuations[43]. In our study, however, we
found that scaling the static nonlinearities to produce a unique solution for the retina’s
impulse response reveals that slow contrast adaptation for system gain does not in fact
exist. The gain changes we observe between low and high contrast conditions are most
likely the same changes predicted by Victor’s instantaneous contrast gain control mecha-
102
Figure 5.6: Changes in gain, timing, DC offset, and spike rate across time
a) The peak of the impulse response, after scaling the associated static nonlinearity, is shown for each two
second period in low (left) and high (right) contrast conditions. The impulse responses are normalized by
the last period in the low contrast condition. Periods are numbered one through five. Values represent
the average gain across all cells. Error bars represent SEM. b) The change in impulse response peak
time, expressed as a percentage of the peak of the impulse response computed in the last period of the
low contrast condition. c) The vertical offset of each two second period, calculated by fitting the static
nonlinearity of each period to the static nonlinearity of the last period in the low contrast condition. d)
Total number of spikes for each period in low and high contrast conditions.
103
nism. These changes are identical in the first and last two second period of each epoch,
which suggests that the contrast adaptation observed in earlier studies may be an artefact
of non-uniqueness. Contrast adaptation may have an effect, however, on the timing changes
associated with this mechanism, since the time to peak in our high contrast condition only
reached steady state by the second two second period. Hence, for the rest of our analysis,
we ignored the effects of contrast adaptation on timing by analyzing the last nine seconds
of each ten second epoch to compute the linear kernels presented above.
5.2
Peripheral Contrast Gain Control
Starting with Kuffler and Barlow’s investigations, descriptions of ganglion cell receptive
fields have focused on their linear center-surround properties[64, 4]. It is universally agreed
that the ganglion cell’s receptive field has an excitatory center and an inhibitory surround.
While this center-surround organization facilitates signal detection in each of the complementary ON and OFF channels, such an organization fails to describe how a ganglion cell is
able to adapt its response sensitivity to a background of peripheral visual signals. Studies
have demonstrated that ganglion cell mean firing rates decrease when a peripheral stimulus
is introduced[39]. More recently, it has been suggested that multiple subunits in the periphery modulate ganglion cell responses, increasing or decreasing firing rate depending on
the spatiotemporal characteristics of the peripheral stimulus[20].
From a signal detection perspective, adjusting the ganglion cell’s linear filter in response
to peripheral stimulation makes sense. In the outer retina, for example, the interaction
between cone and horizontal cell networks keeps cone signals independent of intensity[8],
insuring that ganglion cell responses only encode contrast[102]. This intensity adaptation
adjusts the retina’s dynamic range, enabling it to respond over several decades of mean
104
intensity. We hypothesize that a similar adjustment takes place in the inner retina. In this
case, however, the cellular interactions extend the dynamic range of the retina’s response
to contrast. Hence, introducing a high contrast signal in the periphery moves the center
ganglion cell’s range of contrast sensitivity to higher contrasts, and hence changes its linear
impulse response.
To directly explore the effect of peripheral signals on ganglion cell responses, we again
use our white noise analysis described in Section 3.1 to determine how the linear filter
is affected by stimuli in the periphery. We recorded intracellular responses from guinea
pig retinal ganglion cells as we presented a low contrast white noise sequence with and
without a high contrast drifting grating in the periphery, and measured both the membrane
and spike impulse response under these two conditions. Again, we focus on the impulse
response because we wish to explore the nonlinear effects of the periphery on the retina’s
temporal filter. As mentioned earlier, we restrict our analysis to OFF Y cells since peripheral
stimulation exhibited no significant effect on ON cells.
Our experiment consisted of alternating ten second epochs of a central 10% white noise
sequence with no peripheral stimulation and a central 10% white noise sequence with a
100% contrast 1.33 cyc/deg square wave grating drifting at 2Hz. We ran the experiment for
four minutes and recorded the ganglion cell response. We presented the peripheral stimulus
in the ganglion cell’s far surround, extending from a distance of 0.5 to 4.3 mm out from the
ganglion cell center. We presented the center stimulus as a 500µm spot, centered over the
ganglion cell’s receptive field, whose intensity was governed by the white noise sequence. A
typical ganglion cell response to three of these epochs is shown in Figure 5.7. Introduction
of the peripheral grating causes a slight hyperpolarization of the ganglion cell’s membrane
potential and a decrease in spike rate.
105
Figure 5.7: Recording ganglion cell responses with and without peripheral stimulation
We recorded the ganglion cell response to alternating ten second epochs of a 10% contrast white noise
stimulus while introducing and removing a high contrast drifting square wave in the periphery. We
presented the white noise stimulus as fluctuations in intensity of a 500µm spot centered on the ganglion
cell’s receptive field (top). Recorded responses are shown for three such epochs (no surround signal,
surround signal, no surround signal). For each trace, we extracted the membrane potential, shown in
red, and the spikes to compute both membrane and spike impulse responses.
106
The membrane and spike impulse response computed by cross-correlating the output
with the input are shown in Figure 5.8 for these two conditions. We normalized the curves to
the peak of the impulse responses computed with no peripheral stimulus (NoSurr in figure).
From the figure, we find that when we introduce the peripheral stimulus, the ganglion cell’s
impulse response decreases in magnitude, consistent with the hypothesis that sensitivity
changes with peripheral stimulation. The curves shown in the figure represent the unscaled
impulse responses, and so verifying this change in sensitivity requires making the static
nonlinearities condition-invariant. However, from the raw data we can immediately see
that the surround stimulus has some effect on the gain of the linear kernel, but unlike the
changes we observed when we adjusted depth of modulation, however, we did not observe
a change in the timing of the impulse response.
The membrane and spike static nonlinearities are also shown in Figure 5.8 and we found
that as we introduce the peripheral stimulus, the membrane static nonlinearity reflected
the hyperpolarization observed in the membrane response. We again focused on the linear
impulse response, so we scale the static nonlinearities along the x-axis such that they overlap
one another. In this particular case, to account for the hyperpolarization, we shift the
membrane static nonlinearity along the y-axis before scaling in the x-axis. This step is
reasonable since the shape of the nonlinearity is unaffected by the shift — the displacement
reflects the offset we see in the hyperpolarized response.
We recorded the change in scaled membrane and spike impulse response for the two
stimulus conditions, with and without a stimulus in the far surround, for 14 cells. The
average response, normalized by the peak impulse response computed with no surround,
for both membrane and spike is shown in Figure 5.9a. Shaded regions represent SEM.
As evidenced by the data, introducing a high contrast stimulus in the periphery causes a
consistent gain reduction in the impulse response for both membrane and spikes. Unlike
107
Figure 5.8: Unscaled changes in membrane and spike impulse response and static nonlinearity with peripheral stimulation
For each stimulus condition (10% contrast with and without peripheral stimulation, labeled Surr and
NoSurr), cross-correlation of the ganglion cell membrane and spike response generates the membrane
(top) and spike (bottom) impulse response. The linear prediction of the impulse response, mapped to
the recorded membrane and spike output, generates the static nonlinearly curves shown on the right.
Introducing a peripheral stimulus causes a hyperpolarization in the membrane response which is reflected
in the membrane static nonlinearity. Impulse responses are normalized to the peak of the NoSurr impulse
response, and static nonlinearities are normalized to the peak of the NoSurr static nonlinearity.
108
Figure 5.9: Scaled ganglion cell responses with and without peripheral stimulation
a) The scaled membrane and spike impulse responses computed with and without a high contrast peripheral stimulus for 14 OFF cells. Traces represent average impulse response. Shaded regions represent
SEM, and are colored dark gray for a 10% contrast white noise stimulus without a high contrast peripheral stimulus and light gray for a 10% contrast white noise stimulus with a high contrast peripheral
stimulus. Introducing a peripheral stimulus reduces the system’s gain but does not affect the timing of
the response. b) Root mean squared responses with and without a peripheral stimulus. For each cell, we
averaged the root mean square membrane potential (left) and spike rate (right) across all epochs of each
stimulus condition. Introduction of a peripheral stimulus (solid line on top) causes a decrease in RMS
membrane potential and spike rate that gradually increases over time, while removal of the peripheral
stimulus causes an increase in RMS that gradually decreases. Each cell’s RMS response was fit with a
decaying exponential (gray). The RMS membrane potential and spike rate averaged over all cells is shown
on the bottom. RMS responses are normalized by the peak RMS response, and we express membrane
RMS as fluctuations around the resting potential.
109
increasing stimulus contrast, however, introduction of this square wave grating causes no
appreciable speed up in either the membrane or spike impulse response.
The effect of introducing and removing the high contrast surround grating on ganglion
cell response was also not invariant with time, and had a time course we could measure. To
explore the change in ganglion cell sensitivity with time, we computed the RMS membrane
potential and spike rate for each ten second epoch and averaged across stimulus conditions.
The RMS membrane potential, with resting potential subtracted, is shown with the RMS
spike rate in Figure 5.9b. For one cell, Figure 5.9 shows that when the peripheral stimulus
is removed, both membrane potential and spike rate were initially large, but declined over
time as the cell’s sensitivity decreased. We averaged the RMS responses across all the cells
and found that this behavior was consistent. The time course of this decline was fit a
decaying exponential and we found the time constant for membrane potential to be 2.83
± 0.24 sec and the time constant for spike rate to be 1.89 ± 0.72 sec. When the high
contrast grating was introduced in the periphery, the RMS responses were initially small
but gradually increased as the cell recovered its sensitivity. We also fit this time course with
a decaying exponential for both membrane potential (2.61 ± 0.27 sec) and spike rate (2.89
± 0.63 sec).
From the data, we find that the time constants governing the change in membrane
potential and spike rate with introduction of peripheral stimulus are larger than the time
constants when changing stimulus contrast. We hypothesize that a wide-field amacrine cell
relays signals from peripheral stimuli to affect the center response. Similar to the delay
in conduction through the horizontal cell network[38, 88], signaling through the amacrine
cell network most likely takes some time to flow laterally. Thus, the effect of the peripheral stimulus on RMS response is not immediate, but is determined by lateral conduction
through this network.
110
Because the ganglion cell RMS response changes with time, we explored how the linear kernel and static nonlinearity change with time as we introduced and removed a high
contrast peripheral grating to see if the slow adaptation affected the gain changes we observed. We divided each ten second epoch into five two second periods and used the same
algorithm discussed above. In this case, we set the last two second period of the ten second
epoch without surround stimulation as the reference condition and scaled the membrane
static nonlinearities from the remaining periods to the membrane static nonlinearity from
this reference period. We again concentrated on the change in membrane impulse response
because the transformation from membrane response to spike response is independent of
the experimental condition.
The peak values of the impulse responses measured in the five two second periods for
both low and high contrast conditions, scaled as described above and normalized to the impulse response measured in the last no-surround period, are shown in Figure 5.10a. Without surround stimulation, the peak values are consistent across the five periods and have a
value around 1. When a peripheral grating is introduced, the peak values are also consistent across the five periods, but have a value around 75% of the low contrast peak. This
suggests that peripheral stimulation changes the retina’s gain instantaneously and that this
change persists for the entire ten second epoch. This also suggests that although the effect
of peripheral stimulation does not immediately reach steady state as evidenced by the RMS
observations, the gain change induced by a peripheral grating is immediate. The long time
courses observed in the ganglion cell response may simply govern tonic changes in mean
membrane response and changes in gain.
We also measured how the time-to-peak of the impulse response changes with time when
we switch between the introduction and removal of a high contrast peripheral grating. The
change in peak time, expresses as a percentage change from the peak time of the last period
111
in the no-surround condition, is shown for all periods in Figure 5.10b. Without peripheral
stimulation, the percentage change for all periods is roughly zero, suggesting that the peak
time remains consistent across time. With a peripheral stimulus, however, the percentage
change in peak time fluctuates around 2.5%. This suggests that there may be some timing
change with the introduction of a peripheral stimulus, but this timing change is small. The
fluctuations most likely stem from the fact that each linear kernel is computed with a small
data set (two seconds), and noise in this computation translate to fluctuations in peak
timing changes.
We interpreted the change in vertical offset needed in fitting the static nonlinearities
to the static nonlinearity computed in the last period of the no-surround condition as
a tonic change in mean of the membrane response. This change has no effect on the
gain or timing of the impulse response and tells us how much the membrane depolarizes
or hyperpolarizes in response to peripheral stimulation. We again expressed this vertical
offset as a percentage of the range of membrane responses during that particular period
to compare across periods and across cells. The vertical offsets thus calculated are shown
in Figure 5.10c. We found that when we introduced a high contrast peripheral grating,
the membrane potential immediately hyperpolarized by 25% and slowly increased with
time. By the end of the epoch (with the surround stimulation), the membrane potential
reached a steady state value which was 10% of the range of responses lower than the
corresponding period without surround stimulation. When we removed the grating, the
membrane potential immediately increased by 15% and slowly declined over time. Hence,
as we qualitatively observed in Figure 5.9, introducing and removing peripheral stimulation
has a profound effect on the DC value of the membrane potential which changes with time,
although the change in impulse response gain is persistent across time.
Finally, we recorded the total number of spikes occuring within each two second period
112
Figure 5.10: Changes in gain, timing, DC offset, and spike rate across time
a) The peak of the impulse response, after scaling the associated static nonlinearity, is shown for each
two second period without (left) and with (right) a high contrast peripheral stimulation. The impulse
responses are normalized by the last period in the no-surround condition. Periods are numbered one
through five. Values represent the average gain across all cells. Error bars represent SEM. b) The change
in impulse response peak time, expressed as a percentage of the peak of the impulse response computed in
the last period of the no-surround condition. c) The vertical offset of each two second period, calculated
by fitting the static nonlinearity of each period to the static nonlinearity of the last period in the nosurround condition. d) Total number of spikes for each period with and without peripheral stimulation.
113
to measure how mean spike rate changes across conditions, shown in Figure 5.6d. In general,
the total number of spikes followed the change in mean membrane potential described above.
When we introduced the high contrast peripheral stimulation, the mean spike rate dropped
and slowly recovered over the ten seconds. When the grating was removed, the mean spike
rate immediately rose and then slowly declined over the five two second periods.
Our results suggest that the peripheral stimulation we used, a high spatial frequency, low
temporal frequency drifting grating, causes a consistent, instantaneous, and persistent gain
reduction in the ganglion cell response. The slow changes we observed when we recorded
the membrane and spike RMS responses are associated with a tonic hyperpolarization or
depolarization, probably mediated by a long-range amacrine cell. As mentioned earlier,
Passaglia et al showed that the spatiotemporal nature of a peripheral stimulus determines
whether ganglion cell mean firing rates increase or decrease[20]. They concluded that stimuli
tuned to the X cell receptive field cause Y cell rates to decrease, while stimuli tuned to
the Y cell receptive field cause Y cell rates to increase. In our experiment, our stimulus
represents a low velocity and is tuned to the X cell receptive field. We observe a gain
reduction in our Y cell impulse response, demonstrating that this effect manifests across
all temporal frequencies of the center response. Thus, we conclude that the purpose of
this mechanism is not only to adjust the cell’s contrast dynamic range, but to help higher
cortical structures choose between X and Y channels for extracting visual information —
our peripheral stimulus is tuned to X cells, and so the cortex should pay attention to X cell
signals and ignore our attenuated Y cell responses.
114
5.3
Excitatory subunits
The difference in effects from increasing stimulus contrast and introducing a peripheral stimulus suggests that there are two separate mechanisms that modulate ganglion cell response,
a local mechanism that is responsible for both timing and gain changes, and a peripheral
mechanism that is responsible only for gain changes. If local subcircuits indeed determine
the timing of the ganglion cell response, then we expect there to be an optimal stimulus that
drives this local subcircuit. One candidate for a stimulus that optimally drives the mechanism that drives this local subcircuit is a stimulus that optimally drives the local excitatory
subunits first described by Hochstein and Shapley[49]. Later studies have suggested[38] and
demonstrated[31] that these rectified excitatory subunits are in fact the bipolar cells. Thus,
to explore how these subunits affect the timing of the local subcircuit that is responsible
for timing, we measured how excitation of these subunits change the ganglion cell impulse
response.
We turned again to our white noise analysis to determine how the linear filter is affected
by excitation of these subunits. We recorded intracellular responses from guinea pig retinal
ganglion cells as we presented a low contrast white noise sequence with and without a high
contrast drifting grating, optimized for excitation of the subunits, centered over the ganglion
cell’s receptive field, and measured both the membrane and spike impulse response under
these two conditions. Again, we focus on the impulse response because we wish to explore
the effects of the grating on the retina’s temporal filter. To maintain consistency, we again
only record OFF Y cell responses.
Our experiment consisted of alternating ten second epochs of a 10% white noise sequence
with no central grating and a 10% white noise sequence with a 50% contrast 1.33 cyc/deg
square wave grating, centered over the ganglion cell’s receptive field, drifting at 2Hz for
115
Figure 5.11: Unscaled changes in membrane and spike impulse response and static nonlinearity with central drifting grating
For each stimulus condition (10% contrast with and without a central drifting grating, labeled Grate and
NoGrate), cross-correlation of the ganglion cell membrane and spike response generates the membrane
(top) and spike (bottom) impulse response. The linear prediction of the impulse response, mapped to
the recorded membrane and spike output, generates the static nonlinearly curves shown on the right.
Introducing a high spatial frequency grating over the receptive field center causes an increase in spike
rate, making the spike nonlinearity more linear. Impulse responses are normalized to the peaks of the
impulse response and static nonlinearity.
116
four minutes and recording the ganglion cell response. We presented the center stimulus
as the same 500µm spot, centered over the ganglion cell’s receptive field, whose intensity
was governed by the white noise sequence. We optimized the central grating to elicit
maximum excitation of the excitatory subunits[30], and so introduction of this grating causes
a depolarization in ganglion cell response and an increase in spike rate. The membrane and
spike impulse response computed by cross-correlating the output with the input are shown in
Figure 5.11 for these two conditions. We normalized the curves, which represent the unscaled
linear filters, to the peak of the impulse response computed without the grating. From the
figure, we find that when we introduce the central grating, the ganglion cell’s membrane
impulse response decreases in magnitude and demonstrates a clear shift in timing. To
verify that these changes represent changes in the overall retinal filter, we scaled the static
nonlinearities to make them condition-invariant. In addition, the spike impulse response
increased upon introducing the high contrast central grating, but examination of the spike
static nonlinearity reveals why this may occur.
From Figure 5.11, we see that the increase in spike rate corresponds to a linearization of
the spike static nonlinearity. While scaling between stimulus conditions in the high contrast
and peripheral stimulation experiments entails scaling the x-axis of the static nonlinearity,
in this case, the transformation for spike nonlinearity was not as direct. We could not
find a scaling factor that made the two spike nonlinearities overlap without also allowing
the distribution functions describing the spike nonlinearities to vary in their x-offset (γ
in Equation 3.11). However, allowing γ to vary as a free parameter does not change the
shape of the spike impulse response since γ really only determines spike threshold. In this
experiment, introducing the high contrast drifting grating depolarizes the ganglion cell,
causing a relative reduction in spike threshold and an increase in spike rate. Since the
x-offset, γ, for the spike nonlinearity does not determine the dynamic range of contrast
responses, we still only used the scaling factor governing β (the slope of the nonlinearity)
117
to change the impulse response. Clearly, as seen in Figure 5.11, the slope of the spike
nonlinearity decreases when we introduce the center grating, and so scaling this nonlinearity
to increase the slope translates to a reduction in the gain of the spike impulse response.
We recorded the change in scaled membrane and spike impulse response for the two
stimulus conditions, with and without a drifting central grating, for five cells, four of which
had reasonable spike responses. The average response, normalized by the peak impulse
response with no surround, for both membrane and spike is shown in Figure 5.12a. Shaded
regions represent SEM. As evidenced by the data, introducing a high contrast drifting
grating over the receptive field center causes a consistent gain reduction in the impulse
response for both membrane and spikes. More importantly, however, introduction of this
square wave grating causes a more remarkable timing change in both the membrane or spike
impulse response — the impulse responses computed in the presence of the high contrast,
high spatial frequency central grating are accelerated.
Introduction of the high contrast drifting grating causes an immediate rise in membrane
response and spike rate, as shown in Figure 5.12b, but this increase decayed to baseline
within one to two seconds. We again computed the RMS membrane potential and spike rate
for each ten second epoch and averaged across stimulus conditions. For one cell, Figure 5.12
shows that when the central grating is introduced, both membrane potential and spike rate
were initially large and declined rapidly as the cell’s sensitivity decreased. We averaged the
RMS responses across all the cells and found that this behavior was consistent. We fit the
time course of this decline with a decaying exponential whose time constant for spike rate
is 1.61 ± 0.3 sec and whose time constant for membrane potential is 6.3 ± 2.4 sec. The
membrane time course decays slowly because the difference between the peak and baseline
RMS membrane potential is not large. We also added a second exponential to the fit to
account for the initial rise in response, shown in the figure, although this exponential did not
118
Figure 5.12: Scaled ganglion cell responses with and without a central drifting grating
a) The scaled membrane and spike impulse responses computed with and without a high contrast central
grating for five OFF cells (four for spike response). Traces represent average impulse response. Shaded
regions represent SEM, and are colored dark gray for a 10% contrast white noise stimulus without a
drifting central grating and light gray for a 10% contrast white noise stimulus with a drifting central
grating. Introducing a central grating reduces the system’s gain and accelerates the response. b) Root
mean squared responses to the white noise stimulus with and without a central grating. For each cell,
we averaged the root mean square membrane potential (left) and spike rate (right) across all epochs of
each stimulus condition. Introduction of a high contrast central grating (solid line on top) causes an
increase in RMS membrane potential and spike rate that gradually decreases over time, while removal
of the grating causes a decrease in RMS that gradually increases. Each cell’s RMS response was fit
with a decaying exponential (gray). The RMS membrane potential and spike rate averaged over all cells
is shown on the bottom. RMS responses are normalized by the peak RMS response, and we express
membrane RMS as fluctuations around the resting potential.
119
determine adaptation to the new stimulus condition. When the high contrast grating was
removed from the center, the RMS responses were initially small but gradually increased as
the cell recovered its sensitivity although the change in membrane response and spike rate
after removing the high contrast grating was not as dramatic as the change observed when
we introduced the grating.
5.4
Summary
Each of our three experimental manipulations, increasing stimulus contrast, introducing a
peripheral high contrast grating, and introducing a central high contrast grating, had an
effect on the ganglion cell’s linear impulse response. To quantify the differences in these
effects, we measured the timing and gain changes in the impulse response. For each impulse
response, we measured the peak time, the time at which the impulse response crosses zero,
and the peak time of the second lobe of the biphasic impulse responses. We call these time
points peak, zero, and trough in Figure 5.13a. We express the changes in these time points
as a percentage acceleration from the control condition, which in all cases is the impulse
response computed with the 10% white noise sequence centered over the ganglion cell’s
receptive field. From the figure, we see that introduction of the central grating had the
largest effect on the timing of the impulse response, while introduction of the peripheral
stimulation had minimal effect. Increasing stimulus contrast results in a slight acceleration
of membrane impulse response, but causes a much more pronounced acceleration in spike
impulse response. The retina encodes information in these spikes, and so the acceleration
in spike impulse response has direct implications for visual processing. To quantify the gain
changes, we measure the magnitude of the peak and trough of the impulse response and
normalize these peaks by the peaks in the control condition. In Figure 5.13b, we find that
120
these gain changes are comparable across all three experimental conditions.
The gain and timing of the ganglion cell response are not independent of stimulus conditions, but instead depend on nonlinear interactions that we have attempted to elucidate.
Because stimulation of the ganglion cell center affects both gain and timing, while stimulation in the periphery only affects gain, we suggest that there are two different nonlinear
mechanisms that alter the linear impulse response. A local subcircuit, most likely driven by
the excitatory subunits (or bipolar cells), affects both the gain and timing of the ganglion
cell response. In the periphery, stimulation causes signals to be relayed laterally to the
central local subcircuit, but this information only affects the gain of the response.
To understand some of the precise cellular mechanisms that underly these two mechanisms, we captured preliminary data recorded under different pharmacological conditions.
We hypothesize that the local subcircuit controls both gain and timing of the impulse
response, so we explored the effects of L-2-amino-4-phosphonobutyrate (L-AP4) on the
change in impulse response. Our choice of L-AP4 comes from earlier studies that have
demonstrated the presence of presynaptic metabotropic glutamate receptors (mGluRs) at
the bipolar terminal, that modulate the output of the bipolar to ganglion cell synapse[3].
L-AP4 acts as a competitive agonist of these receptors, and application of L-AP4 to the
bath may potentiate the activity of these receptors. We recorded the ganglion cell impulse
response from a single cell to low and high contrast white noise stimulation with and without L-AP4 and recorded the measured membrane and spike impulse response. As shown in
Figure 5.14a, application of L-AP4 caused both membrane and spike linear kernels to speed
up, suggesting that this synapse may be important in controlling timing information. In
addition, as we switched from low to high contrast without L-AP4, there was a noticeable
timing shift which disappeared when we switched between the two contrast after applying
L-AP4, suggesting that the speed up in the circuit from mGluR activation saturated the
121
Figure 5.13: Comparing gain and timing changes across experimental conditions
a) We measured the time of the peak, zero crossing, and trough of the membrane and spike impulse
response for the three experiments (increasing stimulus contrast, introducing a peripheral stimulus, and
introducing a central grating, which we call ct, surr, and grate in the figure). We express changes in
timing of these three points as a percentage timing reduction compared to the control condition, a 10%
white noise sequence centered over the ganglion cell. Bars represent average percentage change, and
error bars represent SEM. b) We also measured the magnitude of the peak and trough of the impulse
response and normalized these measurements to the magnitude of the peak of the control condition.
Bars represent average normalized change, and error bars represent SEM.
122
timing changes.
While L-AP4 had an effect on the timing of the local subcircuit, it did not affect the
gain changes between the two contrast conditions. Because peripheral stimulation reduced
the gain of the impulse response, and because this information is relayed over relatively
large distances, we hypothesized that this information is carried through spiking amacrine
cells. We applied tetrodotoxin (TTX) to the bath to block Na+ channels, and to therefore block the activity of these spiking amacrine cells. When we changed the white noise
stimulation from low to high contrast, the gain and timing changes were unaffected by
TTX (Figure 5.14), suggesting that the local subcircuit is completely independent of spiking amacrine cells in both gain and timing. However, when we introduced and removed a
peripheral high contrast grating with and without TTX, we found that the gain reduction
from peripheral stimulation was eliminated by TTX (Figure 5.14), confirming that this
information is carried laterally through spiking amacrine cells.
Measurements of the changes in impulse response caused by increasing stimulus contrast, or by introducing a peripheral or central high contrast grating, suggest the presence
of at least two mechanisms by which the ganglion cell changes its linear filter. Our preliminary pharmacological manipulations suggest that we indeed are observing two separate and
distinct mechanisms. Follow up studies that would demonstrate the consistency of these
pharmacological results and that would explore other potential synaptic mechanisms are
necessary to confirm the existence of these two separate mechanisms, and to explain the
cellular interactions underlying these mechanisms.
123
Figure 5.14: Pharmacological manipulations
a) We compared the changes in membrane (left) and spike (right) impulse responses caused by an increase
in stimulus contrast without (top) and with (bottom) L-AP4 applied to the bath. Application of L-AP4
caused an acceleration in both the low and high contrast impulse responses for both membrane and
spikes, but did not affect the relative gain reduction between the two conditions. b) We compared the
changes in membrane impulse response caused by increasing stimulus contrast (left) and by introducing
a peripheral stimulus (right) without (top) and with (bottom) TTX applied to the bath. Application
of TTX caused no remarkable change in the gain reduction when we increased stimulus contrast, but
eliminated the gain reduction when we introduced a peripheral stimulus.
124
Chapter 6
Neuromorphic Models
In the previous chapters, we explored how the retina optimizes its spatiotemporal filters
to encode visual information efficiently. We also observed how the retina adjusts these
filters to adapt to input stimuli, thus maintaining an optimal encoding strategy across a
broad range of stimulus conditions. To gain a better understanding of how the retina
realizes these properties, and to understand how structure and function merge in the design
of such a system, we focus our efforts on developing a simplified model for replicating
retinal processing. Modeling has traditionally been used to gain insight into how a given
system realizes its computations. Efforts to duplicate neural processing take a broad range
of approaches, from neuro-inspiration, on the one end, to neuromorphing, on the other.
Neuro-inspired systems use traditional engineering building blocks and synthesis methods
to realize function. In contrast, neuromorphic systems use neural-like primitives based on
physiology, and connect these elements together based on anatomy[74, 33]. By modeling
both the anatomical interactions found in the retina and the specific functions of these
anatomical elements, we can understand why the retina has adopted its structure and how
125
this structure realizes the stages of visual processing particular to the retina.
In this section, we introduce an anatomically-based model for how the retina processes
visual information. Like the mammalian retina, the model uses five classes of neuronal
elements — three feedforward elements and two lateral elements that communicate at two
plexiform layers — to divide visual processing into several parallel pathways, each of which
efficiently captures specific features of the visual scene. The goal of this approach is to
understand the tradeoffs inherent in the design of a neural circuit. While a simplified
model facilitates our understanding of retinal function, the model is forced to incorporate
additional layers of complexity to realize the fundamental features of retinal processing.
We morphed these neural microcircuits into CMOS (complementary metal-oxide semiconductor) circuits by using single-transistor primitives to realize excitation, inhibition,
conduction, and modulation or shunting (Figure 6.1). In the subthreshold regime, an ntype MOS transistor passes a current from its drain terminal to its source terminal that
increases exponentially with its gate voltage. This current is the superposition of a forward
component that decreases exponentially with the source voltage and a reverse component
that decreases similarly with the drain voltage (i.e., Ids = I0 eκVg (e−Vs − e−Vd ), voltages
in units of UT = 25mV, at 25◦ C[74]; voltage and current signs are reversed for a p-type).
We represented neural activity by currents, which the transistor converts to voltage logarithmically and converts back to current exponentially. Hence, by using the transistor
in three configurations, with one terminal connected to the pre-synaptic node, another to
the post-synaptic node, and a third to modulatory input, we realized divisive inhibition,
multiplicative modulation, and linear conduction.
We use this neuromorphic approach to derive mathematical expressions for the circuits
we use to implement the components of our model and to detail how these circuits are
126
Figure 6.1: Morphing Synapses to Silicon
Circuit primitives for inhibition (left), excitation (middle), and conduction (right). Inhibition: Increased
voltage on the pre-synaptic node (purple) turns on the transistor and sinks more current from the postsynaptic node (green), decreasing its voltage. The voltage applied to the third terminal (modulation, blue)
determines the strength of inhibition. Excitation: Increased voltage on the pre-synaptic node (orange)
turns on the transistor and sources more current onto the post-synaptic node (green), increasing its
voltage. In this case the post-synaptic voltage modulates the current itself, shunting it. We can convert
excitation to inhibition, or vice versa, by reversing either the sign of the pre-synaptic voltage (using a
p-type transistor synapse), the sign of the current (using a current mirror), or the sign of the post-synaptic
voltage (referring it to the positive supply), thereby realizing modulated excitation or shunting inhibition.
Conduction: A bi-directional current flows between the two nodes (brown), whose voltages determine its
forward and reverse components. Both components are modulated by the voltage on the third terminal
(black).
127
connected based on the anatomical interactions found in the mammalian retina. We divide
the retina into two anatomically-based layers, the outer plexiform layer and inner plexiform
layer, and present both the underlying synaptic interactions and the circuit implementations
of these interactions.
6.1
Outer Retina Model
The outer retina transduces light to neural signals, filters these signals, and adapts its gain
locally. Briefly, photons incident on the cone outer segment (CO) cause a hyperpolarization
in the cone terminal (CT) and a decrease in neurotransmitter release from CT. CTs excite
horizontal cells (HC) which provide shunting feedback inhibition on to CT[99]. Both cones
and horizontal cells are coupled electrically through gap junctions[99]. The reciprocal interaction between the cone and HC networks creates a spatiotemporally band-passed signal
at CT. The outer retina’s synaptic interactions are shown in Figure 6.2a.
Our model for the outer retina’s synaptic interactions is shown in Figure 6.2a. By
modeling both the cone and horizontal cell networks as spatial lowpass filters, we can
derive the system block diagram in Figure 6.2b. The system level equations describing
these interactions are:
ihc (ρ) =
A
2
2
(lc ρ + 1) lh2 ρ2 + 1 +
ict (ρ) =
lh2 ρ2 + 1
(lc2 ρ2 + 1) lh2 ρ2 + 1 +
!
A
B
!
A
B
ico
B
(6.1)
ico
B
(6.2)
where B is the attenuation from CO to CT, A is the amplification from CT to HC, and lc
128
Figure 6.2: Outer Retina Model and Neural Microcircuitry
a) Neural circuit: Cone terminals (CT) receive a signal that is proportional to the incident light intensity
from the cone outer segment (CO) and excite horizontal cells (HC). HCs spread their input laterally
through gap junctions, provide shunting inhibition onto CT, and modulate cone coupling and cone
excitation. b) System diagram: Signals travel from CO to CT and on to HC, which provides negative
feedback. Excitation of HC by CT is modulated by HC, which also modulates the attenuation from
CO to CT. These interactions realize local automatic gain control in CT and keep receptive field size
invariant. Both CT and HC form networks, connected through gap junctions, that are governed by their
respective space constants, lc and lh . c) Frequency responses: Both HC and CT lowpass filter input
signals, but because of HC’s larger space constant, lh , HC inhibition eliminates low frequency signals,
yielding a bandpass response in CT. The impulse response associated with CT’s bandpass profile is a
small excitatory central region and a large inhibitory surround.
129
and lh are the cone and horizontal network space constants respectively. HCs have stronger
coupling in our model (i.e. lh is larger than lc ), causing their spatial lowpass filter to
attenuate lower spatial frequencies. Thus, HC lowpass filters the signal while CT bandpass
filters it, as shown in Figure 6.2c, with the same corner frequency, ρA . We can determine this
corner frequency, which corresponds to the peak spatial frequency of the system’s bandpass
filter, by taking the derivative of Equation 6.2 and setting to zero:
∂ict
∂ρ
=
2ico ρ(Alh2 − B(lc + lc lh2 ρ2 )2 )
=0
(A + B(1 + lc2 ρ2 )(1 + lh2 ρ2 ))2
s
ρA = 
1/2
lc
A
− 
B lh
√
1
lc lh
In the case when the horizontal cell network’s space constant is larger than the cone network’s space constant, lh lc , the peak spatial frequency simplifies to
ρA ≈
A
B
1/4
√
1
lc lh
√
which is inversely related to the closed loop space constant lA = (B/A)1/4 lc lh .
In our model, HC activity, which is proportional to intensity, modulates CO to CT
attenuation, B, by changing cone-to-cone conductance, which can adapt cone activity to
different light intensities[11]. However, this local automatic gain control mechanism caused
receptive-field expansion with increased cone-to-cone conductance and undesirable ringing
with high negative feedback gain required to attenuate low-frequencies in earlier designs[8].
To overcome these shortcomings, we complemented HC modulation of cone gap-junctions
130
with HC modulation of cone leakage conductance, through shunting inhibition, making lc
independent of luminance. We also complemented low loop gain with HC modulation of
cone excitation, through autofeedback, thus keeping A proportional to B and fixing ρA .
We choose this gain boosting mechanism since horizontal cells, which release the inhibitory
neurotransmitter GABA, express GABA-gated Cl channels that have a reversal potential of
-20mV. Hence, the Cl channels provide positive feedback and increase the HC time constant
from 65 msec to 500 msec[53].
We can now determine how CT activity depends on these parameters by inserting this
value for ρA into Equation 6.2:
ict (ρA ) =
ico
B(1 + 2lc /lh − lc2 /lh2 )
where we have set A = B to keep lA constant. From the equation, we see that the peak
response depends on the relationship between lh and lc . In the limit where lh lc , the
gain asymptotes to ico /B. Hence, to make CT activity proportional to contrast, we must
set B proportional to local intensity. Hence, we make B equal to HC activity which reflects
intensity. We can derive how this activity changes as we change the relationship between
lh and lc by determining horizontal cell activity, and hence B, at this corner frequency:
B ∝ ihc (ρA ) =
ico
(lh /lc + 2 − lc /lh )
which means
131
ict (ρA ) ∝
lh
lc
for lh lc . This implies that as we change the horizontal cell space constant, lh , we will
change the sensitivity of our outer retina circuit.
We design our outer retina circuit by beginning with the synaptic interactions in Figure 6.2a and formalizing how these interactions can be implemented using current-mode
CMOS primitives. First, we define CT activity as our cone current, Ic . The ratio between
this current and a baseline current, Iu , encodes contrast
Ic
Iu
=
IP
hIP i
where IP represents input photocurrent and hIP i is the spatiotemporal average of this
input. Secondly, we define HC activity as our horizontal cell current Ih and set this equal
to the average light input, hIP i. Hence, multiplying cone activity, Ic /Iu , by HC activity, Ih ,
converts contrast to intensity.
We model the input cones receive from their outer segments, the currents they leak
through their membrane conductance, and the currents they spread through gap junctions.
We also model the excitation horizontal cells receive from cones that is modulated by
autofeedback and the currents they spread laterally through their own network of gap
junctions. We use horizontal cell activity to control the amount of current leaked across
the cone membrane by shunting inhibition and the amount of coupling between cones. If
we assume that each cone and horizontal cell has an associated membrane capacitance, we
can describe these synaptic interactions by the following equations:
132
∂Vh
∂t
∂Vc
Cc
∂t
Ch
Ih
Ic − Ih + αhh ∇2 Ih
Iu
Ic
Ih
= IP − Ih + αcc ∇2 Ic
Iu
Iu
=
(6.3)
(6.4)
where ∇2 ≡ ∂ 2 /∂x2 + ∂ 2 /∂y 2 and represents the continuous approximation of second-order
differences in a discrete network. αcc and αhh are the cone and horizontal cell coupling
strengths respectively, defined as the ratio between the current that spreads laterally and the
current that leaks vertically, with lc ∝ (αcc )1/2 and lh ∝ (αhh )1/2 . Notice that each equation
has an input term, a leakage term, and a spreading term corresponding to the synaptic
interactions described above. Also notice that horizontal cell activity, Ih , modulates both
excitation of horizontal cells by cone activity, Ic /Iu , and cone coupling. Relating these
equations to the block diagram of Figure 6.2, we find that A = B = Ih /Iu as desired. We
shall now construct a CMOS circuit to satisfy these equations.
To modulate cone currents by horizontal cell activity, we use the circuit primitive shown
in Figure 6.3a. Light incident on a phototransistor generates a photocurrent, IP , that discharges Vc . Because this actually corresponds to excitation of the cone node, we define cone
current Ic = I0 e−Vc . We use Vh to determine our horizontal cell current, Ih = I0 eκVh −VL .
Finally, we also define our baseline current, Iu = I0 e−VL , to maintain consistency with our
definition of Ic . From Figure 6.3a, we find:
I = I0 eκVh −Vc = Ih
Ic
Iu
where voltages are in units of UT = 25mV and κ < 1. From Equations 6.3 and 6.4 we
use this current to excite horizontal cells and to inhibit cones as well, which makes the
133
Figure 6.3: Building the Outer Retina Circuit
a) Subcircuit realizing modulation of cone currents (Ic = e−Vc ) by horizontal cell activity (Ih = eVh −VL ).
Solving for I gives an inhibitory current on the cone cell that is equal to Ih Ic /Iu where Iu = I0 e−VL .
b) Coupling between cones is realized through nMOS transistors gated by Vcc and coupling between
horizontal cells is realized through pMOS transistors gated by Vhh .
attenuation from CO to CT equal to the amplification from CT to HC (above).
In addition to the currents between cone and horizontal cell networks, we spread signals
laterally within each network through transistors to model electrical coupling through gap
junctions, as shown in Figure 6.3b. The current from neighboring cone nodes is given by:
αcc ∇2 Ih
Ih
Ic
≈ αcc ∇2 Ic
Iu
Iu
where αcc = eκ(Vcc −Vh ) represents the ratio of current spreading laterally through gap junctions and current drained vertically through membrane leaks. It is exponentially dependent
on the difference in gate voltages, Vcc and Vh . ∇2 Ih Ic /Iu represents the difference in the
differences in current (second derivative), between this node and its neighbors, that is drawn
through the Vc -sourced nMOS transistor (I in Figure 6.3a) by phototransistors. Thus, if
134
two nodes have a large difference in input photocurrent, and if Vcc Vh , then much of this
current difference will diffuse laterally. To make the space-constant, lc , of the cone network
constant, and thus realize receptive-field size invariance, we simply set Vcc = Vh . Adding
this current spread to the input, IP , and subtracting horizontal cell inhibition, Ih Ic /Iu ,
yields Equation 6.4.
To complete Equation 6.3, we also use transistors to implement coupling in the horizontal cell network, as shown in Figure 6.3b. The lateral current between two adjacent
horizontal cell nodes is
0
Ihh = I0 e−κVhh (eVh − eVh )
0
= I0 eVL −κVhh (eVh −VL − eVh −VL )
≈ eVL −κVhh (Ih 0 − Ih )
assuming κ ≈ 1. Therefore, the horizontal cell coupling strength αhh is given by eVL −κVhh ,
giving us the final component of Equation 6.3. Notice that decreasing Vhh increases the
coupling between horizontal cells. Mirroring the modulated cone input, Ih Ic /Iu , back on to
Vh , adding this current to the horizontal cell diffusion current, and subtracting the horizontal
cell current itself produces Equation 6.3.
The complete outer retina CMOS circuit that implements local gain control and spatiotemporal filtering, while using HC modulation and autofeedback to maintain invariant
spatial filtering and temporal stability, is shown in Figure 6.4a for two adjacent nodes.
Photocurrents discharge Vc , increasing CT activity, and excite the HC network through
an nMOS transistor followed by a pMOS current mirror. HC activity, represented by Vh ,
modulates this CT excitation, implementing HC positive autofeedback, and inhibits CT
135
Figure 6.4: Outer Retina Circuitry and Coupling
a) Outer Plexiform Layer circuit. A phototransistor draws current through an nMOS transistor whose
source is tied to Vc and whose gate is tied to Vh . This current, proportional to the product of CT and
HC activity, charges up CT, whose activity is inversely related to voltage Vc , thus modeling HC shunting
inhibition. In addition, this current, mirrored through pMOS transistors, dumps charge on the HC node,
Vh , modeling CT excitation of HC and HC autofeedback. VL sets the mean level of Vc , governing CT
activity. b) Cone coupling is modulated by HC activity. A HC node, Vh in (a), gates three of the six
transistors coupling its CT node (Vc in (a)) to its nearest neighbors.
activity by dumping this same current on to Vc . Cone signals, Vc , are electrically coupled
to the six nearest neighbors through nMOS transistors whose gates are controlled locally
by Vh (Figure 6.4b). These cone signals gate currents feeding into the bipolar cell circuit,
such that increases in Vc , which tracks the level of VL we set, increase the bipolar cell activity. HC signals also communicate with one another, through pMOS transistors, but this
coupling is modulated globally by Vhh , since inter-plexiform cells that adjust horizontal cell
coupling are not present in our chip[59].
136
6.2
On-Off Rectification
To model complementary signaling implemented by bipolar cells, we used the circuit shown
in Figure 6.5b. CT activity is represented by a current, Ic , that we compare to a reference
current, Ir , set by a reference bias, Vref . These currents are inversely related to Vc and
Vref (from our definition above), so we define two new currents, Ic 0 ∝ 1/Ic and Ir 0 ∝ 1/Ir ,
to simplify our understanding of the bipolar circuit. Equating the currents in the current
2 /I
2
mirrors (I1 = Ibq
ON , I3 = Ibq /IOFF ) to the input and output currents, we find
ION + Ic 0 =
2
2
Ibq
Ibq
+
= IOFF + Ir 0
ION IOFF
(6.5)
(6.6)
2 ∝ e−Vbq , which sets the residual current level.
where we have defined the current Ibq
Mirroring the input currents on to one another preserves their differential signal. We set
Ir 0 equal to the mean value of Ic 0 such that the difference is positive when light is brighter
(Ic 0 decreases) and negative when light is dimmer (Ic 0 increases). In practice, we cannot
simply tie Vref to VL because mean CT activity, Vc , is slightly higher than VL because the
drain voltages of the pair of nMOS and pMOS transistors in the outer retina circuit sit at
different levels. We can re-express the relationship between ION and IOFF as:
ION − IOFF = Ir 0 − Ic 0
(6.7)
2 ; the
and we can solve these equations for ION and IOFF as a function of Ic 0 , Ir 0 , and Ibq
result is plotted in Figure 6.5c.
137
Figure 6.5: Bipolar Cell Rectification
a) Signals from CT drive both ON and OFF bipolar pathways. Each bipolar cell half-wave rectifies the
signal, insuring only one pathway is active at any given time. b) Circuit implementation of bipolar cell
rectification. CT activity, Vc , drives a current, Ic , that is compared to a reference current, Ir , driven by
a reference bias, Vref . Both currents are mirrored on to one another, eliminating most of the common
mode (i.e. DC) current and driving subsequent circuitry with the differential signals, ION and IOFF . Vbq
determines the level of residual DC signal present in ION and IOFF . c) The difference between Ic 0 and
Ir 0 determines differential signaling in ION and IOFF (top). When Vc = Vref (i.e. Ic 0 = Ir 0 ), residual
DC currents are proportional to e−Vbq . Directly plotting the difference between cone activity, Ic , and Ir
yields the curves on bottom. Increases in cone activity cause ON currents to saturate while decreases in
cone activity cause OFF currents to increase reciprocally.
138
Since Ic 0 and Ir 0 are both positive, we can determine the common-mode constraint on
ION and IOFF by observing that Equation 6.5 implies
ION , IOFF <
2
Ibq
1
ION
+
1
IOFF
which means
ION + IOFF <
2
2Ibq
ION + IOFF
ION IOFF
2
ION IOFF < 2Ibq
In the case where IOFF Ibq , ION Ibq . Likewise, when ION Ibq , IOFF Ibq .
We can see that the circuit rectifies its inputs around a level determined by Ibq . Hence,
IOFF ≈ Ic 0 − Ir 0 , ION ≈ 0 in the first case and ION ≈ Ir 0 − Ic 0 m IOFF ≈ 0 in the second case.
Hence, as Ic 0 rises above Ir 0 , which reflects less cone activity, current is diverted through the
OFF channel, and as Ic 0 falls below Ir 0 , which reflects more cone activity, current is diverted
through the ON channel (Figure 6.5c).
We can determine the level Iq of ION and IOFF when Ic 0 = Ir 0 = IDC , which represents
the common-mode input current level, from Equation 6.5 as follows:
2
Ibq
Iq
2
2Ibq
IDC
Iq + IDC = 2
⇒ Iq ≈
139
when IDC Iq . Hence, the common-mode rejection in our bipolar circuitry is in fact not
complete, and its outputs contain a residual DC component that is proportional to e−Vbq
and that is inversely proportional to the common-mode input signal, which we set by VL ,
as shown in Figure 6.5c. By lowering Vbq , we can pass more residual current into the inner
retina circuitry and therefore increase baseline activity.
Finally, we can determine how ION and IOFF depend on cone activity, Ic , defined above,
by recalling that Ic ∝ 1/Ic 0 and Ir ∝ 1/Ir 0 . Replotting solutions to the equations derived
above in terms of cone activity yields the curves shown on the bottom in Figure 6.5c. Here,
we can see that as cone activity increases (Vc falls, translating to a rise in Ic ), current is
diverted through the ON channel, but this current level quickly saturates. On the other
hand, as cone activity decreases (Vc rises, translating to a fall in Ic ), current flows through
the OFF channel and increases as the reciprocal of cone activity. Our bipolar circuitry
divides signals into ON and OFF channels, as expected, but the division is not symmetric.
6.3
Inner Retina Model
The inner retina performs lowpass and highpass temporal filtering on signals received from
the outer retina, adjusts its dynamics locally, and drives ganglion cells that transmit these
signals to central structures for further processing[75]. Parasol (also called Y in cat) and
midget (also called X in cat) ganglion cells respond transiently and in a sustained manner,
respectively, at stimulus onset or offset. Both types of ganglion cells receive synaptic input
from bipolar cells and amacrine cells, although Y cells receive more amacrine input (feedforward inhibition)[46, 61]. They also sample the visual scene nine times more sparsely than
X cells, and have proportionately larger receptive fields[24]. Ninety percent of the total
primate ganglion cell population is made up of ON and OFF midget and parasol cells[84]
140
and so we concentrate our modeling efforts on these four cell types.
While the outer retina adapts to light intensity, the inner retina adapts its lowpass
and highpass filters to the contrast and temporal frequency of the input signal. Optimally
encoding signals found in natural scenes requires the retina’s bandpass filters to peak at the
spatial and temporal frequency where input signal power intersects the noise floor[2, 104].
The bandpass filter’s peak frequency remains fixed at this spatial frequency, but increases
linearly with the velocity of the stimulus. We propose that adjustment of loop gain in the
inner retina allow it to adapt to different input power spectra. In addition, as stimulus power
increases, as in the case of increased contrast, optimal filtering demands that the peak of this
bandpass filter move to higher frequencies. The inner retina’s temporal filter exhibits this
adaptation to contrast — ganglion cell responses compress in amplitude and in time when
driven by steps of increasing contrast[107] — by adjusting its time constant[93, 107]. We
propose that these adjustments realized by the inner retina can be accounted for through
wide-field amacrine cell modulation of narrow-field amacrine cell feedback inhibition. Thus,
we offer an anatomical substrate through which earlier dynamic models can be realized.
To realize these functions, we model the inner retina as shown in the block diagram
in Figure 6.6. Bipolar cell (BC) inputs to the inner retina excite ganglion cells (GC), an
electrically coupled network of wide-field amacrine cells (WA), and narrow-field amacrine
cells (NA) that provide feedback inhibition on to the bipolar terminals (BT)[60]. WA,
which receives full-wave rectified excitation from ON and OFF BT and full-wave rectified
inhibition from ON and OFF NA, modulates feedback inhibition from NA to BT. A likely
candidate for WA is the A19 amacrine cell[60] which has thick dendrites, a large axodendritic
field, and couples to other A19 amacrine cells through gap junctions. We use a large
membrane capacitance to model the NA’s slow, sustained, response, which leads to a less
sustained response at the BT through presynaptic inhibition[66]. These BT signals excite
141
both sustained and transient GCs, but transient cells receive feedforward inhibition from
NAs as well[99]. Finally, we hypothesize that a second set of narrow-field amacrine cells
maintains push-pull signaling between complementary ON and OFF channels, ensuring that
only one channel is active at any time. Such complementary interactions between channels
have been demonstrated physiologically through the existence of vertical inhibition between
ON and OFF laminae[86]. Serial inhibition[36] may play a vital role in these interactions.
Our model for the inner retina’s synaptic interactions realizes lowpass and highpass
temporal filtering, adjusts system dynamics in response to input frequency and contrast,
and drives ganglion cell responses. From the block diagram, we can derive the system level
equations for NA and BT, with the help of the Laplace transform:
ina =
g
τA s + ibc , ibt =
ibc
τA s + 1
τA s + 1
(6.8)
where g is the gain of the excitation from BT to NA and where
τA ≡ τna , ≡
1
1 + wg
(6.9)
τna is the time constant of NA and w is the feedback gain determined by WA. From the
equations, we can see that BT highpass filters and NA lowpass filters the signals at BC;
they have the same corner frequency, 1/τA . This closed-loop time constant, τA , depends
on w, and therefore on WA activity. For example, stimulating the inner retina with a
high frequency would cause more BT excitation (highpass response) than NA inhibition
(lowpass response) on to WA. WA activity, and hence w, would subsequently rise, reducing
the closed-loop time constant τA , until the corner frequency 1/τA reaches a point where BT
excitation equaled NA inhibition on WA. This drop in τA , accompanied by a similar drop
142
Figure 6.6: Inner Retina Model
System diagram: Narrow-field amacrine cell (NA) signals represent a low-pass filtered version of bipolar
terminal (BT) signals and provide negative feedback on to the bipolar cell (BC). The wide-field amacrine
cell (WA) network modulates the gain of NA feedback, X. WA receives full-wave rectified excitation from
BT and full-wave rectified inhibition from NA. BT directly drives sustained ganglion cells (GCs) and the
difference between BT and NA drives transient ganglion cells (GCt).
in , will also reduce overall sensitivity.
We can determine the system’s dependence on input contrast by first deriving how the
closed-loop gain wg depends on temporal frequency. Because WA cells are coupled together
through gap junctions, WA activity reflects inputs from BT and NA weighted across spatial
locations. These pooled excitatory and inhibitory inputs should balance when the system
is properly adapted:
w|ina | = |ibt | + isurr
isurr
|ibt |
+
w =
|ina | |ina |
(6.10)
(6.11)
where we define isurr as the current resulting from spatial differences in loop gain values w.
143
|ibt | and |ina | are full-wave rectified versions of ibt and ina , computed by summing ON and
OFF signals. If all different phases are pooled spatially, isurr will cancel out, and w, which
will simply be |ibt |/|ina |, becomes a measure of contrast since it is the ratio of a difference
(highpass signal, ibt ) and a mean (lowpass signal, ina ). From Equation 6.8, we see that in
the DC case, this ratio is equal to 1/g, and the DC gain = 1/2.
The system behavior governed by Equations 6.8 and 6.9 is remarkably similar to the
contrast gain control model proposed by Victor[107], which accounts for response compression in amplitude and in time with increasing contrast. Victor proposed a model for the
inner retina whose highpass filter’s time constant, TS , is determined by a “neural measure
of contrast,” c. The governing equation is:
TS =
T0
1+ cc
1/2
This model’s time constant depends on the neural measure of contrast in much the same
way that our model’s time constant depends on WA activity (Equation 6.9), where Victor’s
T0 is similar to our τna and where Victor’s ratio c/c1/2 is represented by how much WA
activity increases above the DC case in our model. As this activity is sensitive to temporal
contrast, we propose that our WA cells are the anatomical substrate that computes Victor’s
neural measure of contrast.
To explore how our model responds to natural scenes, when the retina is stimulated
by several temporal frequencies simultaneously, we need to understand how the system
computes its loop gain, w, which it determines from the relative contribution of each of
these frequencies. When we stimulate our model with the same spectrum and contrast at
all spatial locations such that there is no difference between surround and center loop gain,
144
we can solve Equation 6.11 for the system’s closed loop gain, w, and find its dependence on
input contrast by setting isurr = 0. Hence, to understand how the system adapts to contrast,
we first must understand the behavior of ibt and ina . We assume sinusoidal inputs, c sin(ωt),
with amplitude c, that are filtered by the outer retina. Hence,
2
jτA ω + 1
jτA ω + 1
jτo ω + 1
2
g
1
=
n0 δ(ω) + c
jτA ω + 1
jτo ω + 1
ibt =
ina
b0 δ(ω) + c
(6.12)
(6.13)
where , τA , and g are defined as above. b0 and n0 are the residual DC activity in ibt and
ina , respectively. From our bipolar circuitry, we find that b0 is determined by Vbq . The
source of NA residual activity, n0 , is explained below. τo is the time constant of the outer
retina’s circuitry, which sharply attenuates frequencies greater than ωo = 1/τo .
A sketch of |ibt | and |ina |’s spectrum is shown in Figure 6.7a. We see that ibt is the
sum of a DC component with amplitude bo , a lowpass component with amplitude c that
cuts off at ωA = 1/τA , and a high pass component that rises as cτA ω, exceeds the lowpass
component at ωn = 1/τna , flattens out at an amplitude of c for frequencies greater than ωA ,
and cuts off at ωo . ina , on the other hand, is the sum of a DC component with amplitude
no and a lowpass component with amplitude cg that cuts off at ωA .
To compute w, we take the ratio of the magnitudes of ibt and ina . Using Parseval’s
relationship, this ratio is the square root of the ratio of the energy contained in ibt ’s spectrum
over that in ina ’s. Simplifying our analysis by setting b0 = n0 and g = 1 and by treating
the temporal cutoffs at ωA and ωo as infinitely steep, we find:
145
Figure 6.7: Effect of Contrast on System Loop Gain
a) BT activity, ibt , is the sum of three components — a DC component that depends on residual BT
activity, b0 , a low pass component that equals c and cuts off at ωA , and a high pass component that
rises as cτA ω, exceeds the lowpass component at ωn = 1/τna , and saturates at ωA . The outer retina
provides an absolute cutoff at ωo . NA activity, ina , is the sum of a DC component that depends on NA
residual activity, n0 , and a low pass filter whose gain is cg and that cuts off at ωA . Loop gain, w, is
determined by the ratio between the energy in ibt and the energy in ina . b) A numerical solution for loop
gain as a function of contrast for three different levels of residual activity, b0 . As b0 increases, the curves
shift to the right, implying that the contrast signal is not as strong. τna is 1.038 sec and τo is 77 msec
for these curves.
146
|ibt |
w=
=
|ina |
b20 +
R ωA 2 2
R ωo 2 !1/2
2
2 2
0 (c + c τA ω )dω + ωA c dω
R ωA
2
b0 +
0
(c)2 dω
(6.14)
Recalling from Equation 6.9 that τA , and hence ωA , and are functions of loop gain w,
we can find a numerical solution to how w depends on contrast. Setting the outer retina
time constant, τo , to 77 msec and the inner retina time constant, τna , to 1.038 sec, we
can determine how w depends on contrast and residual activity, b0 . This relationship is
shown in Figure 6.7b for three different values of b0 . Loop gain w approaches 1 as contrast
approaches 0%. w rises sublinearly with contrast over a range and saturates at a point that
is determined by the amount of residual activity b0 . As b0 increases, the linear regime shifts
to higher contrasts. This implies that the b0 determines the system’s contrast threshold —
higher b0 means that the system needs more input contrast to produce the same loop gain.
This property is analogous to Victor’s c1/2 term from Equation 6.12, whereby the amount
of residual activity determines the strength of the input contrast signal.
The above analysis tells us how the system adjusts its loop gain, and hence time constant,
to input contrast. However, most physiological studies have focused on the retina’s response
when stimulated with only one temporal frequency. We can adopt a similar approach to
characterize our model’s ganglion cell responses, but to do so, we must determine how our
system adapts to a single temporal frequency by deriving a mathematical expression for
w’s dependence on both contrast, c, and input frequency, ω. We use the same approach
as above, where we can express BT and NA activity as a function of the input spectrum
and a residual activity. In this case, however, where we look at the response to a single
frequency, these currents only have energy at DC and at the input frequency. Hence,
Equations 6.12 and 6.13 simplify to the sum of two impulses:
147
jτA ωi + δ(ω − ωi )
jτA ωi + 1
g
= n0 δ(ω) + c
δ(ω − ωi )
jτA ωi + 1
ibt = b0 δ(ω) + c
(6.15)
ina
(6.16)
where , τA , and g are defined as above and ωi is the input frequency. Hence, setting
n0 = gb0 to simplify the computation, we find
s
c2 2 (1 + τna 2 ω 2 ) + b20 (1 + 2 τna 2 ω 2 )
c2 2 + b20 (1 + 2 τna 2 ω 2 )
w =
1
g
w =
1u
t1 +
g
1+
v
u
τna 2 ω 2
b20
c2
1
2
+
b20
τ 2ω2
c2 na
(6.17)
(6.18)
gives the system loop gain as a function of c, g, b0 , and ω, substituting τA = τna . Recall
that ≡ 1/(1 + wg) and which means the loop gain’s 1/g term eliminates the dependence
of , and thus τA , on g.
We can determine how w explicitly depends on c and ω by considering the simple case
where g = 1. This relationship is shown in Figure 6.8a for five different contrast levels,
with a τna of 1 second. Solving Equation 6.18 at different temporal frequencies, we find a
simplified solution for w given by:
w≈



1


 √
ω < 1/τna
2 2
1 + τna ω


q


 1 + c2 /b2
0
1
τna < ω <
ω>
148
c
b0 τna
c 1
b0 τna
Figure 6.8: Change in Loop Gain with Contrast and Input Frequency
The system loop gain, w, depends both on temporal frequency and on contrast. Plots of this relationship
are shown on both a small (left) and large (right) scale. For a given temporal frequency, higher contrasts
generate a larger loop gain. Loop gain rises with temporal frequency, ω, and saturates at a point
determined by the contrast level.
In the DC case, when ω = 0, the system’s loop gain is 1, as expected. Furthermore, we can
see that the loop gain saturates when ω >
c 1
b0 τna .
This point corresponds to higher temporal
frequencies at higher contrasts. Because low contrast curves peel off earlier while higher
contrasts are still relatively close in value, loop-gain increases sublinearly with contrast at
any given temporal frequency. Hence, we can see from the equations that as we increases
stimulus contrast, the system’s adjusts its corner frequency, ωA , such that it increases,
causing a speed up in the ganglion cell response. And finally, since w sets the closedloop time-constant, τA , and tracks ω, the inner retina also effectively adapts to temporal
frequency over the range 1/τna < ω <
c 1
b0 τna .
The adaptation, however, only takes place
over intermediate frequencies — in the DC case (ω = 0), the system’s corner frequency is
set by NA’s time constant and is τna /2 and when ω >
saturates at
c 1
b0 τna ,
the system’s corner frequency
c 1
b0 τna .
BT and NA signals drive ganglion cell responses in our inner retina model. Specifically,
BT signals directly excite both types of ganglion cells (GCs), but transient cells receive feed-
149
forward NA inhibition as well. The system equations determining GCs and GCt responses,
derived from Equations 6.12 and 6.13, as a function of the input to the inner retina, ibc ,
are
jτA ω + ibc
jτA ω + 1
jτA ω + (1 − g)
= b0 (1 − g)δ(ω) + c
ibc
jτA ω + 1
iGCs = b0 δ(ω) + c
(6.19)
iGCt
(6.20)
where we have again made the simplifying assumption that residual NA activity is g times
greater than residual BT activity. In the case when BT to NA excitation has unity gain,
g = 1, feedforward inhibition causes a purely high-pass (transient) response in GCt while
GCs retain a sustained component. With a small loop gain, w, the residual activity, ,
approaches 1/2 and the BT/GCs response approaches an all-pass filter. However, as the
loop gain increases, decreases and the BT/GCs response becomes more highpass. The
change in with loop gain is matched in both BT and NA, and so the difference between
these two signals yields no sustained component in GCt. Thus, GCt produces a purely
highpass response irrespective of the system’s closed-loop gain.
Modulation of NA presynaptic inhibition of BT by WA in the inner retina allows the
circuit to change its closed loop time constant and thereby adjust to different input frequencies. From Equation 6.19, we find that for low frequencies, ω < 1/τna , the GCs response
simplifies to c/2 since ≈ 1/2 over this whole region. As frequencies rise above ω = 1/τna ,
we expect the GCs response to rise and saturate at c at ω = 1/τA — but as temporal
frequency increases, loop gain adaptation occurs and 1/τA progressively increases, since we
1
<ω <
can assume τA ω ≈ 1 for τna
c 1
b0 τna
because of the way w tracks ω in this regime.
Therefore, the rise in GCs is offset by this shift in corner frequency 1/τA , and the entire
150
GCs response is flat for these temporal frequencies. Hence, we expect that GCs’ response
will be unaffected by changes in τna since GCs’ response is flat across all ω. Finally, we
expect GCs responses to remain independent of g, since the term does not appear in the
equations.
From Equation 6.20, we can determine how the GCt response changes with different
input temporal frequencies. For low temporal frequencies, ω < 1/τna , GCt depends on a
lowpass term that is 21 (1 − g) and a term that rises with temporal frequency with a slope
1
determined by contrast. At intermediate temporal frequencies, τna
< ω <
c 1
b0 τna ,
GCt
responses saturate at a level determined exclusively by contrast. Reducing τna will shift the
onset of this saturation range to higher frequencies. Furthermore, although g does not affect
the temporal dynamics of the GC response, we can see from the equations that increasing
g will attenuate low frequency responses in GCt.
The above analysis demonstrates that the interaction between open-loop time constant
τna , temporal frequency ω, and contrast c determines frequency response when we stimulate
with a single temporal frequency. GCt responses rise linearly with ω at frequencies below
1/τna and become flat at high temporal frequencies, while GCs responses are flat in both
regions as τA adapts to ω. At temporal frequencies above
c 1
b0 τna ,
adaptation saturates at
a point determined by stimulus contrast — the corner frequency here will have little effect
on system dynamics since the ganglion cell response in this region will be flat.
The responses of the different inner retina cell types in this model to a step input
is shown in Figure 6.9. BC activity is a low-pass filtered version of light input to the
outer retina. Increase in BC causes an increase in BT and a much slower increase in NA.
The difference between BT and NA activity determines WA activity, which modulates NA
feedback inhibition on to BT. Thus, after a unit step input, BT activity initially rises
151
Figure 6.9: Inner Retina Model Simulation
Numerical solution to inner retina model with a unit step input of 1V. Traces show 1 second of ON cell
responses for the bipolar cell (BC), bipolar terminal (BT), narrow-field amacrine cell (NA), wide-field
amacrine cell (WA), transient ganglion cell (GCt), and sustained ganglion cell (GCs). WA receives input
from cells in ON and OFF pathways. Outer retina time constant, τo , is 96 msec; τna is 1 second.
but NA inhibition, setting in later, attenuates this rise until BT activity is equaled by
gain-modulated NA activity. WA represents our measure of contrast and receives full-wave
rectified input from BT and NA and thus rises above its baseline value of 1 for both step on
and step off. BT drives the sustained GC response, GCs, which persists for the duration of
the step while the difference between BT and NA activity drives the transient GC response,
GCt.
Because the system’s response to a single input frequency is flat from 1/τna to
c 1
b0 τna ,
a
dramatic effect on its response profiles is only obtained when the system is driven by more
than one frequency. WA adapts τA to an individual input frequency. By itself, this change
produces only a minimal change in the ganglion cell response. When multiple frequencies
are present, as in the case of natural vision, however, WA will attempt to adapt to all of
152
them simultaneously and its state will reflect their weighted average. As we showed earlier,
this could explain the temporal aspect of contrast gain control, as frequency weighting is
contrast-dependent. Sensitivity to all frequencies drops when stimulus contrast increases,
but low frequency gains are attenuated more[93]. The differential effect of contrast can
be measured by simultaneously stimulating the retina with the sum of several sinusoids,
approximating a white noise stimulus. In this case, for any individual frequency, WA
activity will not reflect what the adapted activity for that individual frequency ought to
be. This could cause low frequency responses to be attenuated more than high frequency
responses when stimulus contrast increases, generating the contrast gain control effect.
In addition, WA activity also reflects inputs weighted across spatial locations, and is
determined by differences in center and surround WA activity. We can determine the
contribution from different loop gains at different spatial locations by taking into account
the resistance of the WA network in Equation 6.11, with isurr = ∆w/R, where ∆w is the
difference between loop gain in a surround location, ws , and a center location, wc and R is
the resistance coupling these two locations. Thus, if loop gain in the surround is larger than
that in the center, we expect the loop gain in the center to increase, whereas if the opposite
is true, we expect loop gain in the center to decrease. From Equation 6.11, we can solve
for center loop gain wc ’s dependence on the WA coupling, which we defined as resistance
R, and surround WA activity, ws . We find
wc =
R|ibt |c + ws
R|ina |c + 1
(6.21)
where loop gain depends on BT and NA activity in the center. Similarly, we can determine
the loop gain in the surround by a reciprocal relationship
153
ws =
R|ibt |s + wc
R|ina |s + 1
(6.22)
where surround loop gain depends on BT and NA activity in the surround. In both cases,
since |ibt | ≥ |ina |, as we increase the resistance of the network, R, loop gain at that location
becomes more dependent on that location’s BT and NA activity. Through WA coupling,
loop gain is determined by averaging loop gain across the network, and so this isolation can
cause either an increase or a decrease in loop gain, depending on how the spatial average
relates to the center loop gain. This makes sense since each location’s loop gain becomes
more isolated from the rest of the network as we increase resistance. When we decrease
resistance, WA activity is distributed throughout the network, and at any given location,
is more or less than in the isolated condition, depending on the relative values of the loop
gain at different spatial locations (unless all locations are computing the same loop gain, in
which case WA activity is unchanged). We therefore expect that if we increase WA network
resistance, we will make the center ganglion cell response depend more on the loop gain
computed at the center. Furthermore, we know that the loop gain at a given location tracks
the temporal frequency of input at that location. Hence, we also expect different temporal
frequencies in the surround to have different effects on the center loop gain.
6.4
Current-Mode ON-OFF Temporal Filter
The synaptic interactions that implement our inner retina model are shown in Figure 6.10a.
We synthesize our inner retina circuit by beginning with these synaptic interactions and the
block diagram shown in Figure 6.6a, deriving the differential equations that govern these
interactions, and formalizing how these interactions can be implemented using currentmode CMOS primitives. First, we define the equation for NA’s lowpass response to input
154
Figure 6.10: Inner Retina Synaptic Interactions and Subcircuits
a) ON and OFF bipolar cells (BC) relay cone signals to ganglion cells (GC), and excite narrow- and
wide-field amacrine cells (NA, WA). NAs inhibit bipolar cells (BT), WAs, and transient GCs in the
inner plexiform layer; their inhibition onto WAs is shunting. WAs modulate NA presynaptic inhibition
and spread their signals laterally through gap junctions. BTs also excite local interneurons that inhibit
complementary BTs and NAs. b) Subcircuit used to excite NA with I2 = (Iτ /(In+ +In− ))It+ . c) Subcircuit
used to inhibit NA with I1 = (Iτ /(In+ + In− ))In+ or I2 = (Iτ /(In+ + In− ))In− .
BT signals
τna
∂In
= It − In
∂t
(6.23)
where τna is the time constant of NA, In is NA activity, and It is BT activity, and where
activity is represented by currents in this current-mode CMOS circuit. To implement complementary signaling, we represent all signals differentially. Thus, Equation 6.23 becomes
τna
∂(In+ − In− )
= (It+ − It− ) − (In+ − In− )
∂t
(6.24)
where In+ and It+ are the ON NA and BT currents and In− and It− are the OFF NA and BT
155
currents. In subthreshold, these currents are an exponential function of their gate voltages
+
(i.e. In+ = I0 eκVn
/UT )
τna
and so Equation 6.24 becomes
∂V −
κ + ∂Vn+
− In− n ) = (It+ − It− ) − (In+ − In− )
(In
UT
∂t
∂t
(6.25)
Secondly, we assume that ON and OFF NA activity is limited by a geometric mean constraint. Thus, the product of their currents must remain constant and equal to Iq2 which
sets quiescent NA activity. This relationship is also governed by its own time constant, τc ,
and so we derive the second equation for our filter
τc
∂In+ In−
= Iq2 − In+ In−
∂t
Expanding this equation and using the same subthreshold voltage-current relationship as
above, we find that
τc
Iq2
κ ∂Vn− ∂Vn+
(
+
)= + − −1
UT ∂t
∂t
In In
(6.26)
If we express both τna and τc in terms of membrane capacitance and leakage currents
(τna =
Cn UT
κIn ,
τc =
Cc UT
κIc ),
Equations 6.25 and 6.26 become
∂V −
Cn + ∂Vn+
− In− n ) = (It+ − It− ) − (In+ − In− )
(In
In
∂t
∂t
−
Iq2
Cc ∂Vn
∂Vn+
(
+
) =
−1
Ic ∂t
∂t
In+ In−
156
(6.27)
(6.28)
Substituting Equation 6.28 into Equation 6.27, we find that
∂V +
Cn +
Ic Cn 2 +
(In + In− ) n =
(I /I − In− ) + (It+ − It− ) − (In+ − In− )
In
∂t
In Cc q n
If we assume that the two time constants, τna and τc , are equal, we can take advantage of
the fact that Ic /Cc = In /Cn . Thus, we define Cn = Cc = C and In = Ic = Iτ where C
and Iτ determine NA’s time constant for both common-mode and differential signals. The
equation then simplifies to
C
∂Vn+
Iτ
[(I + − It− ) − (In+ − Iq2 /In+ )]
= +
∂t
In + In− t
(6.29)
This equation tells us the currents used to charge and discharge the positive NA capacitor.
Similarly,
C
∂Vn−
Iτ
= +
[(I − − It+ ) − (In− − Iq2 /In− )]
∂t
In + In− t
(6.30)
determines how the negative NA capacitor is charged and discharged. A CMOS circuit
that is described by Equations 6.29 and 6.30 will realize the computations needed for NA
activity in our push-pull model. By dividing these equations into two terms that charge
or discharge the NA capacitors (i.e. Vn+ and Vn− ), we can derive the subcircuits that will
realize these computations.
Starting with the first term on the right of the equations, we construct the subcircuit
shown in Figure 6.10b. Current entering this subcircuit, It+ , comes from ION in the bipolar
157
circuit of Figure 6.5. Vτ s modulates this current through a tilted nMOS mirror that generates the current I1 . For simplicity, we ignore κ and express all voltages in units of the
thermal voltage, UT . Thus,
I1 = It+ eVτ s −V1
By setting this current, I1 equal to the sum of the positive and negative NA currents, In+
and In− , we can compute a current I2 in Figure 6.10b that is equal to the first term in
Equations 6.29 and 6.30. Specifically,
I2 = I0 eV1 −VS =
It+ I0 eVτ s −VS
In+ + In−
By setting Vτ s = VS + Vτ , the current I2 , which we use to charge up Vn+ , equals It+ Iτ /(In+ +
In− ). A complementary circuit on the negative side of the circuit generates a current
It− Iτ /(In+ + In− ). Taking the difference between these two currents with a current mirror yields the first terms of Equations 6.29 and 6.30. Thus, the current charging up the
positive NA capacitor is
C
∂Vn+
Iτ
(I + − It− )
= +
∂t
In + In− t
(6.31)
The first part of the second term of Equations 6.29 and 6.30 represents a leakage current
from the NA capacitors. To realize this computation, we implement a current divider that
links positive and negative sides of the circuit, as shown in Figure 6.10c. The current drawn
158
+
through both sides of the pair, Iτ , is eVn
−V
−
+ eVn
−V .
Hence, the current on one side of the
current correlator, I1 , is
+
I1 = eVn −V
Iτ In+
=
In+ + In−
This current drains charge away from the positive NA capacitor, and a complementary
current drains charge from the negative NA capacitor. Hence, first part of the second term
of Equations 6.29 and 6.30 is satisfied:
C
Iτ
∂Vn+
=− +
I+
∂t
In + In− n
(6.32)
Finally, the second term of Equations 6.29 and 6.30 include a second part that is dependent on the quiescent activity, Iq2 , which determines total NA activity by charging both
NA capacitors. This determines NA’s residual activity, n0 , discussed above. The subcircuit
that realizes this term is shown in Figure 6.11a. Current through the nMOS transistor
gated by Vb is equal to the sum of the positive and negative NA currents. Hence
eV1 =
I0 eVb
+
In + In−
This node, V1 , gates two nMOS transistors that dump current back on to the NA capacitors
(Vn+ and Vn− ). This current on the positive side is given by
159
Figure 6.11: Inner Retina Subcircuits
a) Subcircuit used to excite NA with (Iτ /(In+ + In− ))(Iq2 /In+ ). b) Subcircuit that realizes WA modulation
of NA feedback inhibition on to BT.
+
I1 = I0 eV1 −Vn =
I0 2 eVb −V +
e n
In+ + In−
If we set Vb = Vq + VS + Vτ , then this current charging Vn+ becomes
I1 =
Iτ I0 2 1 Vq
e
In+ + In− In+
By defining the current Iq2 as I0 2 eVq , this current satisfies the third term of Equation 6.29:
C
Iq2
∂Vn+
Iτ
= +
∂t
In + In− In+
(6.33)
and a complementary current charges the negative NA capacitor. Combining the three
subcircuits satisfies Equations 6.29 and 6.30.
160
Thus far, these equations only compute BT to NA excitation in our inner retina model.
To implement NA feedback inhibition on to BT, modulated by WA, we use the subcircuit
shown in Figure 6.11b. The voltage at node V represents WA activity and is the source
of a transistor gated by Vn+ . Thus, this activity modulates NA feedback inhibition on to
BT — as voltage increases, gain, w, goes down and as voltage decreases, gain increases.
Furthermore, WA activity at this node changes with BT excitation and NA inhibition. V
decreases with increased current in It+ and It− (not shown), thus realizing excitation of WA
activity (increased gain), and increases with increased current in In+ and In− (not shown),
thus realizing shunting inhibition of WA activity. Finally, WA nodes are coupled to one
another through an nMOS diffusion network gated by Vaa . This voltage determines the
strength of WA coupling, and this voltage determines the resistance R of Equation 6.11
through a simple relationship, R ∝ eκVaa . As By adding this subcircuit, we can close the
feedback loop in our inner retina model.
Finally, we use inner retina circuitry to drive ganglion cell responses. In the ON pathway,
a copy of the BT signal, It+ , drives an ON sustained ganglion cell. The difference between
an additional copy of It+ and a copy of In+ drives an ON transient ganglion cell. In fact, our
circuit generates two copies of ON transient signal so that we can pool transient ganglion
cell inputs over larger areas (see below). Because we divide current from the ON bipolar
cell into five copies of It+ (three for ganglion cells, one to excite WA, and one to excite
NA), we compensate for this reduction in WA excitation by driving WA with five copies
of It+ produced by the nMOS mirror shown in Figure 6.11b. All of these interactions are
reproduced on the negative side of our inner retina circuit, producing the final inner retina
shown in Figure 6.12.
Because we have control over both Vaa and Vτ s , we can explore how changing the dynamics of the system changes ganglion cell responses. WA activity, which modulates inhibitory
161
Figure 6.12: Complete Inner Retina Circuit
The complete inner retina circuit is shown with different subcircuits boxed out. Red dash represents
the subcircuit shown in Figure 6.10b, green dash represents the subcircuit shown in Figure 6.10c, blue
dash represents the subcircuit shown in Figure 6.11a, and cyan dash represents the subcircuit shown in
Figure 6.11b.
162
NA feedback onto BT, is distributed throughout the array by a network of Vaa -gated nMOS
transistors. Because WA modulation determines the dynamics of GC responses, we expect
the extent of spatial coupling in the WA network, controlled by Vaa , to affect circuit dynamics. In addition, the relationship between Vτ s , VS , and Vτ determine the DC loop gain
of the system. Ideally, Vτ s should be set equal to VS + Vτ for a DC loop gain of one (see
above). If Vτ s > VS +Vτ , then the DC loop gain is greater than one, circuit dynamics should
be faster, and GCt responses should be inhibited. However, if Vτ s < VS + Vτ , then the DC
loop gain is less than one, causing the opposite effects.
The remaining biases in the inner retina circuit are important for the circuit to operate
correctly, but should have little effect on the dynamics of GC responses. Vbq determines
residual current passed to the inner retina from BC and therefore determines quiescent
GC activity. VS acts as a virtual ground for the NA subcircuit. Thus, WA activity can
be represented by voltage deviations below VS . Total NA activity is controlled by Vb as
discussed above. Finally, we added a bias Vos for the source of the two pMOS transistors
used to mirror Iτ It+ /(In+ + In− ) on to the positive NA capacitor (we added the same bias
on the negative side as well). This keeps the drain voltages of these transistors similar,
insuring that excitation on to one NA capacitor is matched by equal inhibition from the
complementary side.
Finally, analog signals in the mammalian retina cannot be relayed over long distances,
mammalian ganglion cells use spikes to communicate with higher cortical structures. Similarly, each GC in the chip array receives input from the inner retina circuit and converts
this input to spikes, as shown in Figure 6.13a. Our silicon neurons translate current into
spikes and exhibit spike-rate adaptation through Ca++ activated K+ channel analogs[50].
The CMOS circuit that realizes this transformation is shown in Figure 6.13b. Briefly, input current charges up a GC membrane capacitor. As the membrane voltage approaches
163
Figure 6.13: Spike Generation
a) Input current to the ganglion cell produces a spike that is conveyed down the optic nerve. Spike rate
is a function of input current. b) A CMOS circuit that transforms input current to spikes. Iin from
the inner retina charges up a GC membrane capacitor. When the membrane voltage crosses threshold,
the circuit produces a spike (Sp) that is relayed off chip by digital circuitry. This circuitry acknowledges
receipt of the spike by sending a reset pulse (RST) that discharges the membrane and dumps charge on
a current-mirror-integrator that implements Ca++ spike-rate adaptation.
164
threshold, a positive feedback loop, modulated by Vfb , speeds the membrane’s approach to
threshold. Once threshold is passed, the circuit generates a pulse (a spike) that is relayed
to digital circuitry. The digital circuitry acknowledges receipt of the spike by sending a
reset pulse which discharges the membrane. The reset pulse, RST, also dumps a quanta of
charge on to a current-mirror integrator through a pMOS transistor gated by Vw . Charge
accumulating on the integrator models the build-up of Ca++ within the cell after spikes.
This charge, which leaks away with a time constant determined by Vτ n , draws current away
from the membrane potential, modeling Ca++ mediated K+ channels. The virtual ground
for the neuron circuit, VSn , is set to be the same as the virtual ground for the inner retina
circuit, VS .
6.5
Summary
The CMOS circuits described above extract contrast signals from visual scenes and spatiotemporally filter these signals to generate four parallel representations of visual information. Our model realizes luminance adaptation, bandpass spatiotemporal filtering, and
contrast gain control. In the outer retina, cone membrane capacitances and gap-junction
coupling attenuate high temporal and spatial frequencies while feedback inhibition from the
horizontal cell network, which has larger membrane capacitances and stronger gap-junction
coupling, attenuates low temporal and spatial frequencies. The outer retina adjusts to input luminance through horizontal-cell modulation of cone gap-junction coupling and cone
excitation (autofeedback). These interactions generate a cone terminal signal that is proportional to contrast and a horizontal cell signal that is proportional to mean luminance.
Signals emerging from the outer retina are rectified into complementary ON and OFF channels by bipolar cells. This ensures an efficient push-pull architecture that allows separate
165
pathways to dedicate their entire channel capacity to coding their respective signals. In
the inner retina, we implemented narrow-field amacrine cell feedback inhibition to generate
a high pass temporal response in the bipolar terminal. A wide-field amacrine cell, which
computes signal contrast, modulates this inhibition and hence changes the dynamics of the
bipolar terminal response. We use the bipolar terminal response to drive sustained-type
ganglion cells, and we use feedforward inhibition from narrow-field amacrine cells to remove
the residual component of the bipolar response in driving transient-type ganglion cells.
The information theoretic explorations outlined earlier, and the physiological demonstration of the retina’s ability to adjust its temporal filters described later, suggest that
any valid model of retinal processing needs to maintain the ability to adapt to input stimulus. Our model presented in this chapter seems to satisfy this requirement — we expect
the CMOS circuit to maintain the same response profile over a large range of mean light
intensities, we expect the inner retina circuitry to adjust the systems corner frequency such
that it tracks the temporal frequency of the input stimulus, and we expect the wide-field
amacrine cell, which computes contrast, to adapt the closed-loop system gain, and hence
change the response profile of the circuit’s outputs.
166
Chapter 7
Chip Testing and Results
In the previous chapter, we described a simplified model based on the retina’s anatomy
and physiology that replicates retinal processing. In this model, coupled photodetectors
(cf., cones) drive coupled lateral elements (horizontal cells) that feed back negatively to
cause luminance adaptation and bandpass spatiotemporal filtering. Second order elements
(bipolar cells) divide this contrast signal into ON and OFF components, which drive another
class of narrow or wide lateral elements (amacrine cells) that feed back negatively to cause
contrast adaptation and highpass temporal filtering. These filtered signals drive four types
of output elements (ganglion cells): ON and OFF mosaics of both densely tiled narrow-field
elements that give sustained responses and sparsely tiled wide-field elements that respond
transiently.
Our motivation for morphing these neural circuits in silicon is to attempt to duplicate
the brain’s computational power. The neuromorphic approach has been most successfully
applied in the retina[67], whose physiology and anatomy are known in great detail. These
167
pioneering efforts realized logarithmic luminance encoding and highpass spatiotemporal filtering by replicating the function of the three cell types in the outer retina. Later attempts
realized a fixed-receptor field size and bandpass spatiotemporal filtering by extending the
cell types modeled to bipolar and amacrine cells[8]. We have extended the neuromorphic
approach further by incorporating the ganglion cell layer in our model and by implementing a novel push-pull architecture. By morphing a total of thirteen cell types in both the
inner and outer retina, we have implemented luminance adaptation, bandpass spatiotemporal filtering, and contrast gain control. Our chip’s outputs are coded as spike trains on
four parallel pathways that replicate the wide-field, transient and narrow-field, sustained
ganglion cells[108], found in both ON and OFF varieties[64] in all mammalian retinas. In
primates, these four types give rise to ninety percent of the axons in the optic nerve[84].
Similar to the mammalian retina, our retinomorphic chip realizes visual sensory processing using three layers of neuron-like elements[36], connected in a parallel feedforward
architecture, and two classes of interneuron-like elements, which provide local inhibitory
feedback[99]. A schematic of all the synaptic interactions found in our outer and inner
retina model is shown in Figure 7.1. To implement spatiotemporal bandpass filtering, chip
inter-cone gap junctions and membrane capacitances attenuate high frequencies while chip
horizontal cells, which have larger membrane capacitance and stronger gap junction coupling, inhibit the cones and remove low frequencies. To realize luminance adaptation, chip
horizontal cells shunt current across the cone membrane and modulate cone gap-junctions,
making cone sensitivity inversely proportional to luminance. The horizontal cell activity
reflects average luminance since they use autofeedback, found in tiger salamander horizontal
cells[53], to boost excitation from the cone’s contrast signal. To implement complementary
signaling and nonlinear spatial summation, chip bipolar cells rectify signals into ON and OFF
channels[38]. Chip bipolar cells and amacrine cells also receive inhibition from the complementary channel, similar to vertical inhibition between ON and OFF laminae[86] and serial
168
inhibition found between mammalian amacrine cells[36], ensuring that only one channel is
active at any time. To create a transient ganglion cell response, chip narrow-field amacrine
cells inhibit ganglion cells, like in mammalian retina[99], canceling out the sustained bipolar
inputs they receive. They also inhibit the bipolar terminal, as demonstrated in salamander retina1[66], and chip wide-field amacrine cells modulate this inhibition, changing the
dynamics and gain of the bipolar response to realize contrast gain control. Chip wide-field
amacrine cells directly measure contrast since they are excited by highpass ON and OFF
bipolar cells, whose activity represents the difference between the signal and the mean, and
inhibited by lowpass ON and OFF narrow-field amacrine cells, whose activity represents the
mean. Finally, we convert analog inputs to spikes at the ganglion cell level using a pulsegenerating circuit with spike-rate adaptation. This chapter describes our retinomorphic
chip and shows that its four outputs compare favorably to the four corresponding retinal
ganglion cell types in spatial scale, temporal response, adaptation properties, and filtering
characteristics.
7.1
Chip Architecture
The CMOS circuits described in Chapter 6 extract contrast signals from visual scenes and
spatiotemporally filter these signals to generate four parallel representations of visual information: OnT (ON transient), OnS (ON sustained), OffT (OFF transient), and OffS (OFF
sustained). Our chip implements the mammalian retina’s architecture at a similar scale.
The chip has 5760 photoreceptors at a density of 722 per mm2 and 3600 ganglion cells at a
density of 461 per mm2 — tiled in 2×48×30 and 2×24×15 mosaics of sustained and transient ON and OFF ganglion cells. A portion of our chip layout is shown in Figure 7.2a. The
distance between adjacent chip photoreceptors, which are 10 µm on a side and hexagonally
169
Figure 7.1: Retinal Structure
Chip cone terminals (CT) receive a signal that is proportional to incident light intensity from the cone
outer segment (CO) and excite horizontal cells (HC). Horizontal cells spread their input laterally through
gap junctions, provide shunting inhibition onto cone terminals, and modulate cone coupling and cone
excitation. ON and OFF bipolar cells (BC) relay cone signals to ganglion cells (GC), and excite narrowand wide-field amacrine cells (NA, WA). Narrow-field amacrine cells inhibit bipolar terminals, wide-field
amacrine cells, and transient ganglion cells; their inhibition onto wide-field amacrine cells is shunting.
Wide-field amacrine cells modulate presynaptic inhibition and spread their signals laterally through gap
junctions. Bipolars also excite local interneurons that inhibit complementary bipolars and amacrine cells.
170
tiled like the cone mosaic, is 40 µm, which is only about two and a half times the distance
between neighboring human cones at 5 mm nasal eccentricity[25]. Unlike neural tissue,
silicon microfabrication technology can only produce planar structures, so post-synaptic
circuitry must be interspersed between the photoreceptors. Each pixel contains a phototransistor, outer retina circuitry, bipolar cells, and one-quarter of the inner retina circuit.
Hence, four adjacent pixels are needed to generate the four ganglion cell type outputs. Because transient ganglion cells occur at a lower resolution, not every pixel contains ganglion
cell spike-generating circuitry. Three out of every eight pixels instead contain the large NA
membrane capacitor described in Chapter 6. A pulse generating circuit, also described in
Chapter 6, in the remaining five pixels converts GC inputs into spikes that are sent off chip.
Mammalian retina exhibits convergence of cone signals on to bipolar cells[99], which
makes the receptive field center Gaussian-like[96]. To implement such convergence in our
model, chip bipolar cells connect the outputs from a central phototransistor and its six
nearest neighbors (hexagonally tiled) to one inner retina circuit, as shown in Figure 7.2b,
and have a dendritic field diameter of 80µm. Thus, Vc , which represents CT activity in the
outer retina circuit (Figure 6.4) in fact drives two nMOS transistors in the BC circuit (only
one is shown in Figure 6.5). A central photoreceptor drives BC with the output of both
of these transistors while photoreceptors at the six vertices divide these outputs between
their two nearest BCs. For symmetry, we implement a similar architecture for the reference
current driven by Vref .
Because we modeled our chip transient cells after cat Y-ganglion cells, we wanted to
replicate the receptive field size and nonlinearities exhibited by these ganglion cells. Y
cells pool their inputs from a large receptive field and this pooling accounts for the Y-cell
nonlinear subunits[38]. Each inner retina circuit described above generates two copies of
transient GC input, for both ON and OFF pathways. We maintain hexagonal architecture
171
Figure 7.2: Chip Architecture and Layout
a) 2×3-pixel array of chip layout compared to human photoreceptor mosaic: The large green squares,
which are floating bases of CMOS-compatible vertical bipolar-junction transistors, transduce light into
current. Each pixel, with 38 transistors on average, has a photoreceptor (P), outer plexiform layer (OPL)
circuitry, bipolar cells (BC), and inner plexiform layer (IPL) circuitry. Spike-generating ganglion cells
(GC) are found in five out of eight pixels; the remaining three contain a narrow field amacrine (NA)
cell membrane capacitor. Inset: Tangential view of human photoreceptor mosaic at 5mm eccentricity
in the nasal retina. Large profiles are cones and small profiles are rods (taken from [25]). b) Chip
Signal Convergence: Signals from a central photoreceptor (not shown) and its six neighbors are pooled
to provide synaptic input to each bipolar cell (BC). Each bipolar cell generates a rectified output, either
ON or OFF , that drives a local IPL circuit. Sustained ganglion cells receive input from a single local
IPL circuit. Signals from a central IPL circuit (not shown) and its six neighbors are pooled to drive each
transient ganglion cell.
172
at the level of the inner retina, although local inner retina circuits are tiled at one-quarter
the density of the phototransistors that provide their input. Therefore, we employ a similar
scheme for pooling inner retina signals — whereas one inner retina microcircuit, whose
input represents the synaptic drive of one bipolar cell, drives a sustained ganglion cell, the
outputs of seven neighboring inner retina microcircuits are pooled to drive one transient
ganglion cell, which has a dendritic field diameter of 240 µm. The central inner retina
circuit excites its ganglion cell with both copies of its transient output whereas inner retina
circuits at the six vertices divide these outputs between their two nearest transient GCs.
This architecture, shown in Figure 7.2b, creates a transient GC response that has a large
receptive field with spatial nonlinear summation.
Because of wiring limitations, we can not directly communicate each GC output off chip,
an asynchronous address-event transmitter interface reads out spikes from each pixel[9].
Each GC interfaces with digital circuitry that communicates the spikes to an arbiter at the
end of each row and each column of neurons, as shown in Figure 7.3. The arbiter multiplexes
incoming spikes and outputs the location, or address, of each spiking neuron as they occur.
X and Y addresses for each GC are communicated serially off chip. Hence, we can represent
the activity of all 3600 ganglion cells with just seven bits. By noting the address of each
event generated by the chip, we can decode ganglion cell type and location in the array.
We designed and fabricated a 96×60 photoreceptor 3.5×3.3mm2 chip in 0.35µm CMOS
technology. Our silicon chip generates spike train outputs for four prototypical ganglion-cell
types that we name OnT (ON-Transient), OnS (ON-Sustained), OffT (OFF-Transient), and
OffS (OFF-Sustained). The chip’s light response to a drifting vertical sinusoidal grating
is shown in Figure 7.4. Our chip’s four ganglion cell type outputs are color coded in
the figure: OnT (blue), OnS (green), OffT (yellow), and OffS (red). Spike trains from
identical GCs in a single column of the chip array differ significantly (OnT spike rate CV
173
Figure 7.3: Spike Arbitration
A GC communicates spikes to peripheral digital handshaking and arbitration circuitry using row and
column request lines. The arbiter chooses between spikes by selecting a row and column, encodes each
incoming spike into a pair of seven-bit addresses, and communicates these addresses off chip. X and
Y addresses are sent serially on the same address bus; a multiplexer between row and column arbiters
toggles between row and column bits. In addition, the handshaking circuits relay reset signals back to
the spiking GC to reset its membrane voltage.
174
Figure 7.4: Chip Response to Drifting Sinusoid
A raster plot of the spikes (top) and histogram (bottom, bin width = 20 msec) recorded from a single
column of the chip array. The stimulus was a 3Hz 50%-contrast drifting sinusoidal grating (0.14cyc/deg)
whose luminance varied horizontally across the screen and was constant in the vertical direction. We
use a 50% contrast stimulus in all responses presented here unless otherwise noted. GC outputs are
color-coded as shown in the legend. We computed the amplitude of the fundamental Fourier component
of these histograms, which is plotted in all frequency responses presented here, unless otherwise noted.
The same applies to physiological data reproduced for comparison.
(coefficient of variation) = 57%, OnS = 162%) due to variability between nominally identical
transistors (mismatch). To get a robust measure of their activity, we average responses
for each type over the entire column and analyze the spike histogram. These histograms
demonstrate phase differences between the four GC types: complementary ON and OFF
channels respond out of phase with one another while transient cells lead sustained cells,
exhibiting both earlier onset and shorter duration of firing.
175
7.2
Outer Retina Testing and Results
Our outer retina circuit’s nonlinear behavior generates CT signals that are entirely proportional to contrast. Because of local automatic gain control, we expected the circuit’s
contrast signals to be independent of incident light intensity. Mammalian retina exhibits
this behavior, where ganglion cell responses are dependent on signal contrast and not on
signal mean light intensity[102], and we hypothesize that this light adaptation takes place
in the outer retina. The OnT ganglion cell responses shown in the Figure 7.5 maintain this
contrast sensitivity over at least one and a half decades of mean luminance (our experimental
setup was limited to ∼200 cd/m2 ).
Our outer retina circuit, however, is not ideal, and we had to compensate for this shortcoming. We found that decreasing incident light intensity, thereby decreasing photocurrents
passing through the outer retina circuit, caused a decrease in Vh which caused Ih to decrease
more slowly than Ihh (because κ <1). Specifically, the horizontal cell space constant αhh
depends on the ratio between lateral and vertical currents in the horizontal cell network. If
we focus on how changing Vh changes αhh , we find that
αhh =
Ihh
Ih
e−κVhh +Vh
eκVh
(1−κ)Vh
∝ e
∝
The outer retina’s closed loop space constant lA is proportional to
√
tional to αhh . Hence, lA , is described by
lA ∝ (e(1−κ)Vh )1/4
176
√
lc lh and lh is propor-
(7.1)
Figure 7.5: Luminance Adaptation
a) Cat ON-center Y-cell responses to a sinusoidal grating (0.2cyc/deg) whose contrast varied between 1
and 50% and reversed at 2 Hz, for five mean luminances. Mean luminance is converted from trolands
to cd/m2 based on a 5 mm diameter pupil (adapted from [102]). c) Chip OnT cell responses to a
sinusoidal grating (0.22 cyc/deg) whose contrast varied between 3.25 and 50% and reversed at 3 Hz,
for four different mean luminances. In a and b, response versus contrast (small x-axis) curves are shifted
horizontally according to mean luminance (large x-axis) such that the 50% contrast response is aligned
with that particular mean luminance.
177
As discussed in Section 6.1, we set horizontal cell activity, Ih = eκVh , equal to the average
light input, hIP i. Thus, Vh is determined by mean light intensity:
eVh ∝ hIP i1/κ
Inserting this term into Equation 7.1, we find that
ρA = 1/lA ∝ hIP i
−(1−κ)
4κ
For a κ equal to 0.7, for example, this dependence becomes hIP i−0.1 . In other words, the
system’s spatial frequency shifts to lower frequencies with increasing mean light intensity,
although the effect will be minimal. In summary, decreasing incident light intensity, and
therefore decreasing photocurrents passing through the outer retina circuit, causes a decrease in Vh which causes the vertical current Ih to decrease more slowly than the lateral
current Ihh (because κ <1). This results in a smaller horizontal space constant, αhh , and a
slightly larger ρA .
To explore this effect, we measured GC spatial profiles in response to a 7.5 Hz drifting
sinusoid (50% contrast) of different spatial frequencies as we changed mean intensity (Figure 7.6 top). Because ρA is only weakly dependent on mean intensity, we found that the
normalized spatial profiles for both OFF sustained and transient ganglion cell responses to
be essentially independent of intensity, as expected. Decreasing intensity, however, had a
slight effect on OFF cell sensitivity; the effect was larger for ON cell sensitivity. OffT GC
peak responses, which were 408 sp/s, 432 sp/s and 218 sp/s as we decreased mean intensity
178
from 196 cd/m2 to 33 cd/m2 to 3.3 cd/m2 , are relatively unchanged with intensity until
mean intensity drops to 3.3 cd/m2 . For OnT GC’s, the peak response dropped from 771
sp/s to 485 sp/s to 117 sp/s as we decreased mean intensity from 196 cd/m2 to 33 cd/m2
to 3.3 cd/m2 .
To understand how changing intensity affects system gain, we revert to the outer retina
system equation and determine how the gain of these equations is affected by changes in
intensity. First, from above, we can express the horizontal cell network space constant lh
as a function of mean intensity. Thus,
lh ∝ hIP i
1−κ
2κ
In the case where we are reducing mean light intensity, we are also reducing the value of
the horizontal cell space constant lh , albeit slowly. From Section 6.1, we found that CT
sensitivity is proportional to lh . However, the horizontal cell space constant is only weakly
related to mean intensity (∝ hIP i0.214 from above for κ = 0.7), and because sensitivity is
proportional to lh , this change with intensity can not completely account for the drop in
peak response. As we lower mean intensity from 196 cd/m2 to 33 cd/m2 to 3.3 cd/m2 , we
expect the ganglion cell response to drop by 31.7% followed by an additional drop of 38.9%.
Our data shows that the OFF response initially does not drop and then drops by 49.5%
while the ON response initially drops by 37% and then drops again by 75.9%. The effect
of lh can only account for the initial drop in the ON ganglion cell response, but there is no
corresponding drop in the OFF ganglion cell response. Furthermore, the change in lh does
not account for the drop in response in either channel as we lower intensity from 33 cd/m2
to 3.3 cd/m2 . This suggests that the drop in sensitivity may arise from another nonlinearity
in the circuit.
179
Figure 7.6: Chip Response to Drifting Sinusoids of Different Mean Intensities
Responses of chip OffS and OffT cells to 7.5Hz horizontally drifting sinusoids with different spatial
frequencies. Normalized responses recorded at three different mean luminances without changing Vhh
to compensate for the outer retina nonideality are shown on top. Normalized responses recorded while
changing both mean luminance and Vhh are shown on bottom. Vhh values are in units of mV.
180
Reexamining the circuit diagram in Figure 6.4a, we see that incident light creates a
photocurrent, IP , that is drawn through an nMOS transistor and excites the HC network,
whose activity is determined by Vh , through a pMOS current mirror. When light intensity
drops, the amount of photocurrent decreases. This causes the drain voltage of the nMOS
transistor to sit higher (since less current is flowing through the p-mirror) and Vh to sit
lower (since less current from the p-mirror will translate to lower currents through the diodeconnected nMOS). Because current through the nMOS, which is determined by IP , increases
more with the rise in drain voltage than it decreases with the fall in the gate voltage Vh ,
since κ < 1, Vc rises to compensate. This distorts the rectification in our bipolar circuit that
requires Vc = Vref . The rise in Vc means more current is diverted through the OFF channel
than through the ON channel. From the data, we can see that this makes sense. Both ON
and OFF sensitivity decrease, but ON sensitivity decreases more. The overall reduction in
sensitivity for both channels most likely arises from parasitic effects that contribute to the
ganglion cell response. Mean spike activity is affected by stray photocurrents in general,
which determine the time constant for spike rate adaptation in the spike-generating neuron.
As intensity drops, and therefore as these photocurrents decrease, the time constant of this
adaptation increases, causing a drop in spike rate and overall activity.
From Figure 7.6 top we also find that the OFF transient peak response lies at 0.1644
cyc/deg whereas the corresponding ON transient peak response lies at 0.1096 cyc/deg (not
shown). This implies that the OFF channel has a smaller space constant lA than the ON
channel. Both channels, however, are driven by the same outer retina circuitry, so this
difference most likely arises from asymmetric rectification in the bipolar circuit. Currents
diverted to the ON channel in the bipolar circuit saturate, whereas currents in the OFF
channel increase as Vc rises above Vref . Saturation in the ON channel changes the spatial
tuning of the ON ganglion cell response. The receptive field of both ON and OFF ganglion
cells can be described by a Mexican hat spatial structure — each ganglion cell has a narrow
181
excitatory center and a broader inhibitory surround. The width of this Mexican hat is
determined by the system’s spatial corner frequency, ρA . Saturation of the ON channel
reduces the peak of the center response, which we can interpret as a relative increase in
the width of the excitatory center. This translates to a decrease in the ON channels corner
spatial frequency, ρA , which is what we observe in the data.
To compensate for the change in sensitivity, we manually decreased Vhh , which increases
αhh , to boost CT activity. In the above analysis, this has the effect of increasing lh , which
increases ict since it is proportional to lh . To determine how much we should change Vhh
to compensate for the change in sensitivity, we recorded the ganglion cell response at one
spatial and temporal frequency and adjusted Vhh to keep this response fixed at different
mean intensities. Because we did not measure how the ganglion cell’s entire spatiotemporal
response changed with changes in Vhh , this technique only gives us an estimate for how much
we should change Vhh to compensate for changes in intensity. For every decade reduction
in photocurrent, we had to decrease Vhh by 85mV to maintain the same response at this
spatial and temporal frequency. If this change in Vhh completely accounted for changes in
the ganglion cell response by changing the relative current levels passing through the HC
coupling transistors, these numbers would correspond to a κ of 0.548. However, this low
value of κ suggests that although we wanted to compensate for the nonideality discussed
above by retaining the same level of inter-horizontal cell coupling (i.e. the same Ihh /Ih ratio
in Figure 6.3b) at different intensities, we overcompensated for this nonideality in order to
maintain sensitivity at this spatial and temporal frequency.
Overcompensating for the outer retina nonideality by decreasing Vhh further has the
effect of expanding the receptive field size, or lowering ρA , similar to the expansion observed
in mammalian retina at lower light intensities[52]. In Figure 7.6 (bottom), we measured
GC spatial responses at different mean intensities while compensating for sensitivity by
182
changing Vhh . From Section 6.1, we found that αhh and lh also depend on κVhh . Since the
√
outer retina’s closed loop space constant lA is proportional to lc lh , its dependence on Vhh
is described by
ρA =
1
∝ (eκVhh )1/4
lA
For κ = 0.7, decreasing Vhh from 15mV to -40mV, for example, should cause a 28% reduction
in ρA , ignoring the negligible change due to intensity. We found that the peak spatial
frequency of the transient OFF ganglion cell in fact decreased from 0.2192 to 0.1644 cyc/deg
as we decreased Vhh , corresponding to a reduction of 25%. The change in spatial profile
for both OFF transient and sustained ganglion cell responses for further reductions in Vhh
at different mean intensities is shown in Figure 7.6 (bottom). As expected, decreasing Vhh
caused the peak spatial frequency to decrease, expanding the ganglion cell’s receptive field.
7.3
Inner Retina Testing and Results
Chip ganglion cells spatially bandpass filter visual signals, with chip transient cells displaying nonlinear spatial summation similar to that found in wide-field transient responding
mammalian cells[38]. Chip transient cells’ spatial-frequency sensitivity peaks at a lower
spatial frequency (0.22 cyc/deg) than chip sustained cells (0.33 cyc/deg) (Figure 7.7a), as
expected from their larger receptive fields. When we varied the phase of the sinusoidal grating, and reversed its contrast at 7.5 Hz, the second Fourier component (F2) of the sustained
cells’ response disappeared at certain phases while this component could not be nulled in
the transient cells (Figure 7.7a). This difference, the same fundamental distinction found
183
between narrow- and wide-field mammalian ganglion cells[40], arises because, whereas a
single bipolar cell drives the sustained cell, several drive the transient cell. When the blackwhite border of the grating is centered over a bipolar cell, its net photoreceptor input does
not change, nulling the sustained cell’s response. However, all bipolar cell signals feeding
into a transient cell cannot be nulled simultaneously, and these signals cannot cancel out
each other because they are rectified. The fluctuations in its F2 response with spatial phase
arise from uneven spatial sampling in our chip.
Chip transient cells, like cat wide-field transient cells, retain a band-pass response to
temporal frequencies at all spatial frequencies while chip sustained cells, like cat narrowfield sustained cells, are band-pass at low spatial frequencies but become low-pass as spatial
frequency increases (Figure 7.7c,d)[47]. This transformation occurs in chip sustained cells
because horizontal cell inhibition is ineffective at high spatial frequencies, as most of their
excitatory input is lost to neighboring horizontal cells through gap-junctions, while chip
transient cells retain their bandpass response because feedforward narrow-field amacrine
inhibition suppresses low temporal frequencies. Chip sustained cells also capture the overall
suppression of low to intermediate temporal frequencies seen in cat narrow-field sustained
cell measurements at low spatial frequencies, which we believe arises from increased widefield amacrine cell activity at these spatial frequencies. However, they do not reproduce the
rapid increase at high temporal frequencies, presumably because the wide-field amacrine
signal does not roll-off early enough.
The current-mode CMOS inner retina circuit performs highpass and lowpass temporal
filtering on input signals and adjusts the time constants of these filters by adapting to
input frequencies. The dynamics of the circuit are governed by NA’s time constant which
is determined by the size of the NA capacitor and by Iτ , which effectively drains these
capacitors. Because space is at a premium in VLSI design, we restricted the size of our
184
Figure 7.7: Spatiotemporal filtering
a) Responses of chip OnT and OnS cells to 7.5 Hz horizontally drifting sinusoids with different spatial
frequencies. b) Amplitude of the second Fourier component of OffT and OffS ganglion cells in response
to a 0.33cyc/deg contrast-reversing grating at different spatial phases. c) Responses of cat ON-center
Y-cells (left) and chip OnT cells (right) to low, medium, and high spatial-frequency sinusoidal gratings
drifting horizontally at different temporal frequencies. d) Same as c, but for cat ON-center X-cells (left)
and chip OffS cells (right). (Cat data is reproduced from [47]).
185
NA capacitors to 1 pF. This leaves little room for the magnitude of Iτ if we are to expect
reasonable circuit dynamics. In testing the chip, we found that we had to set Vτ at 50mV to
attain reasonable responses (see below). However, this makes Iτ susceptible to stray leakage
currents generated in the substrate by incident photons. In fact, increasing incident light
intensity causes photocurrents to dominate Iτ , placing an upper limit on light intensity given
the NA capacitor sizes we used. Therefore, in addition to decreasing Vhh to compensate for
nonlinearities at lower light levels, we also had to increase Vτ to maintain the same level of
Iτ .
Because GCt’s corner frequency depends on τna , as we found in Equations ?? and ??,
we expect that increasing Vτ will reduce τna and therefore shift GCt’s temporal profile to
higher frequencies. We also expect that GCs’ response will be unaffected by changes in
Vτ since it asymptotes to the same value when ω 1/τna and ω 1/τna . To verify this
prediction, we measured GC responses to a 0.2192 cyc/deg drifting sinusoidal grating at
different temporal frequencies and recorded the temporal profile for different levels of Vτ .
To ensure that we were only adjusting τna , however, we had to compensate for changes in
Vτ with changes in Vτ s since we had to set Vτ s = Vτ + VS to keep BT to NA excitation,
g, equal to 1. As we increased Vτ from 10mV to 50mV to 90mV, while also increasing
Vτ s to keep g = 1, the peak ON GCt response was 386 sp/s, 297 sp/s, and 418 sp/s while
the peak ON GCs response was 319 sp/s, 271 sp/s, and 310 sp/s respectively. The peak
values were relatively constant, suggesting that we compensated for changes in Vτ with
appropriate changes in Vτ s , and so we normalized the peak response for each level of Vτ to
focus on changes in peak temporal frequency. As shown in Figure 7.8, increasing Vτ caused
low frequency GCt responses to be attenuated, as the system’s corner frequency increased.
GCt responses shown in the figure are bandpass because GCt produces a purely highpass
version of signals at BC, which represents a lowpass filtered version of light signals. The
outer retina’s time constant which determines the corner frequency of its lowpass filter is
186
Figure 7.8: Changes in Open Loop Time Constant τna
Temporal frequency responses of chip OnT and OnS ganglion cells to a 0.2192 cyc/deg drifting sinusoidal
grating. Profiles are shown for three different values of Vτ , which determines the open loop time constant.
Increasing Vτ causes a decrease in τna , thus increasing the system’s corner frequency. To compensate for
changes in DC loop gain, we also changed Vτ s , whose values are also given. Phase data are shown on
the bottom.
clearly smaller than the inner retina’s closed loop time constant, generating the bandpass
response. GCs responses represent an all-pass version of signals at BC, and are therefore
dominated by the outer retina’s lowpass filter. Changing Vτ has no effect on GCs temporal
frequency profile.
Increasing Vτ s effectively increases the gain of BT to NA excitation, and therefore introduces an arbitrary open-loop gain into the system. Because τA ≡ τna /(1 + wg), the
closed-loop time constant is unaffected by g since w ∝ 1/g. This means that g will have
187
no effect on the temporal dynamics of our ganglion cell responses. From Equation ??, we
found that GCs responses remain independent of temporal frequency, ω, and its response
is still dominated by the outer retina circuit’s lowpass time constant. Furthermore, from
Equation ??, we see that increasing g will additionally attenuate low frequency responses
in GCt by introducing a DC component that is determined by 1 − g (in the limit where
ω 1/τna , the arbitrary gain g contributes a component that is (1 + c)(1 − g)/2). In addition, also from Equation ??, increasing g will attenuate high frequency responses in GCt,
but the amount of this attenuation falls as contrast increases. Intuitively, one can imagine
that increasing g will increase NA activity, which provides more feedforward inhibition on
to GCt. To verify this, we measured the temporal profile of GC outputs in response to a
50% contrast 0.2192 cyc/deg drifting sinusoidal grating for different levels of Vτ s . In this
case, Vτ s0 is 620mV and Vτ is 50mV. At this contrast level, as we increased Vτ s from 560mV
to 620mV to 680mV, the peak ON GCt response dropped from 1600 sp/s to 568 sp/s to 117
sp/s while the peak ON GCs response remained relatively unchanged and was 261 sp/s, 377
sp/s, and 338 sp/s respectively. To focus on g’s effect on the systems temporal dynamics,
we plotted the normalized responses, shown in Figure 7.9. Increasing Vτ s above 620mV
causes an attenuation in low frequency GCt responses while lowering Vτ s makes the low
frequency roll-off less severe. There was little effect on the high frequency responses since
the 50% input contrast mitigated the attenuation in this region. As expected, introducing
the open loop gain g into the system had little effect on GCs responses since these responses
were independent of g.
As discussed above, WA activity encodes contrast, and therefore allows contrast to
change system timing and gain. When presented with contrast-reversing square-wave gratings of increasing contrast, our chip’s transient cell exhibited contrast gain control[93].
Their responses increased sublinearly and became more transient (Figure 7.10b), similar to
the behavior observed in cat narrow-field sustained and wide-field transient ganglion cells
188
Figure 7.9: Changes in Open Loop Gain g
Temporal frequency responses of chip OnT and OnS ganglion cells to a 0.2192 cyc/deg drifting sinusoidal
grating. Profiles are shown for three different values of Vτ s , which determines the open loop gain.
Increasing Vτ s above the DC unity value of 620mV introduces an arbitrary gain term into Equation ??,
increasing the system’s corner frequency. Phase data are shown on the bottom for the three different
Vτ s conditions.
189
Figure 7.10: Contrast Gain Control
a) Responses of a cat ON-center X-cell to a 1 Hz square-wave contrast reversal of a 1 cyc/deg sinusoidal
grating at four different peak stimulus contrasts (C). Bin width for spike histograms is 3.7 msec (reproduced from [107]). b) Responses of chip OnT (top) and OffT (bottom) cells to a 1 Hz square-wave
contrast reversal of a 0.22 cyc/deg grating at the same contrasts. Spike rates are the average for an
entire column. Bin width is 4 msec.
(Figure 7.10b)[107]. The time constant of the response’s decay dropped from 28 to 22 msec
as contrast increased from 6.25% to 50%.
To better quantify the effect of increasing contrast, we measured the chip’s temporal
frequency sensitivity in response to a 0.14 cyc/deg contrast reversing sinusoidal grating
whose temporal modulation signal was the sum of eight sinusoids. The temporal frequencies
of the input were 0.214 Hz, 0.458 Hz, 0.946 Hz, 1.923 Hz, 3.876 Hz, 7.782 Hz, 15.594 Hz,
and 31.219 Hz. These frequencies are identical to those chosen in Victor’s demonstration
of contrast gain control[93] and were chosen to minimize higher order interactions. We
presented the stimulus at four input contrasts: 1.25%, 2.5%, 5%, and 10%. Our chip
OffT cells shift their sensitivity profile to higher temporal frequencies as contrast increases
190
from 1.25% to 10% (Figure 7.11a). In addition, as contrast increased, response amplitude
saturated. As contrast gain control occurs at the bipolar terminal, we expected to observe
its effects in sustained cells as well. However, contrast gain control was not as dramatic in
these cells, suggesting that narrow-field amacrine cell feed-forward inhibition enhances its
effects.
To verify that the contrast changes we observed in the transient ganglion cell responses
were consistent with our model, we fit the curves in Figure 7.11a with the system equations
derived above. We introduced a sinusoidal input of contrast c to a simplified model of our
system. The outer retina is approximated by a lowpass temporal filter with time constant
τo , whose output drives a transient ganglion cell response that is the difference between
Equation 6.12 and Equation 6.13. Thus, the ganglion cell response, in spikes per second, is
given by
2
jτA ω + (1 − g) 1
GCt = S b + c
jτA ω + 1
jτo ω + 1 !
1
jτp ω + 1 where τA ≡ τna and where = 1/(1 + w). b in the equation is the residual GCt activity, determined by the difference between residual ibt activity b0 and residual NA activity
n0 . We also introduced a term that models the lowpass filtering behavior of the chip’s
photoreceptors whose time constant is τp . We fit the four curves by allowing the system
gain, S, the loop gain, w, and the residual GCt activity, b, to vary across different stimulus
contrasts and by fixing the remaining parameters. The best fits of this model to the four
input contrasts are shown as the solid lines in Figure 7.11a. We found that the parameters
that fit these curves best were τp = 33 msec, τo = 77 msec, τna = 1.0382 sec, and g = 1.07.
The residual activity, b, increased monotonically from 0.0038 to 0.02 as we increased contrast from 1.25% to 10%. Although our initial model for how the system adjusts its corner
191
frequency in response to contrast was premised on a fixed level of both ibt and ina residual
activity, we found that our values for b were still on the same order of magnitude as the
values of b0 we used to generate the curves in Figure 6.7.
As expected, as stimulus contrast increases, the system’s loop gain also increased. The
best fits for loop gain, w, in the four contrast conditions are shown in Figure 7.11b. As
contrast increases by a factor of eight, the loop gain increases by a factor of 3.5. This
demonstrates the same behavior we had predicted by Equation 6.14 and showed in Figure 6.7. In fact, using a value of b0 = 0.017 in Equation 6.14 generates a dependence of
loop gain, w, on contrast that closely approximates the dependence we see in Figure 7.11b
at high contrasts. The discrepancy between the fixed level of residual activity, b0 , we use in
Equation 6.14 and the increasing level of residual activity, b, we use to fit our data suggests
that residual activity in fact has a dependence on contrast — increased signal power causes
an increased level of quiescent activity. Furthermore, as stimulus contrast increases, the
system gain, S, that best fits our data saturates, as shown in Figure 7.11c, demonstrating
the contrast gain control mechanism’s gain compression. Our prediction for how our system
computes signal contrast fits the data quite well, suggesting that we have implemented a
valid model for contrast gain control.
WA activity encodes a neural measure of contrast that determines the system’s loop
gain by modulating NA feedback inhibition of BT signals. We expected the extent of
spatial coupling in the WA network to affect circuit dynamics. However, changing Vaa , and
thus changing the WA coupling, by itself had no remarkable effects on circuit dynamics
(data not shown). WA signals, however, did induce gain changes in GC responses when we
examined spatial interactions of different input signals. To demonstrate this, we stimulated
a single column of the chip array with a sinusoidally modulated 50% contrast 4.56◦ bar,
which matched the columns receptive field, and measured the temporal frequency profile
192
Figure 7.11: Change in Temporal Frequency Profiles with Contrast
a) Chip Off-Transient cell response to a 0.14 cyc/deg contrast reversing sinusoidal grating whose temporal
modulation signal was a sum of eight sinusoids. The amplitude of the fundamental Fourier component
at seven of the eight frequencies for four different modulation contrasts is shown. Solid lines are the
best fit of an analytical model of the chip circuitry. b) The loop gain that best fit Equation 7.2 increases
as stimulus contrast increases. The behavior is similar to our prediction for change in loop gain, shown
in Figure 6.7. (c) As stimulus contrast increases, the system gain that best fits the data saturates,
suggesting that contrast gain control causes a reduction in ganglion cell sensitivity.
193
of the response. We then introduced a 0.11 cyc/deg square-wave grating in the column’s
surround and remeasured the temporal frequency profile of the column’s center response. We
chose this spatial frequency since it produced the greatest effect on the center ganglion cell
response. The effect of both a 2 Hz and 7.5 Hz surround grating was the same on transient
and sustained cells — the surround signal caused a reduction in sensitivity without affecting
the temporal profile (Figure 7.12a). This suggests that the WA network sets a loop gain, ws ,
that is determined by the temporal frequency of the surround stimulation. This loop gain
increases the effective loop gain computed in the center without surround stimulation since
both 2 Hz and 7.5 Hz gratings generate a loop gain, ws , that is greater than that generated
with no surround stimulation. The WA network modulates system gain by relaying these
signals laterally and increasing the effective center loop gain, reducing the sensitivity of the
center.
To explore how spatial coupling in the WA network modulates these lateral gain changes,
we measured the effect of changing Vaa on the column’s center response. We stimulated the
same column with a 5 Hz sinusoidally modulated 50% contrast 4.56◦ bar and recorded the
response’s F1 amplitude with no surround stimulus as we changed Vaa . We then introduced
a 0.11 cyc/deg square-wave grating drifting at 2 and 7.5 Hz and measured how the center’s
F1 amplitude changed with different values of Vaa . Figure 7.12b demonstrates the change
in amplitude for OffT and OffS ganglion cells for different values of Vaa , with and without
surround stimulation. Our experimental protocol was such that we set a value of Vaa and
then recorded the ganglion cell response in the three conditions — no surround stimulus,
followed by a 2 Hz and 7.5 Hz drifting grating — before changing Vaa . Hence, we can ignore
any changes in chip activity over time by comparing the relative values of the ganglion cell
response under these three conditions for each value of Vaa . The non-monotonic behavior
shown in the curves probably reflects these changes in underlying chip activity, and we
focus on the relative values of the three curves for the purpose of this analysis. Decreasing
194
Figure 7.12: Effect of WA Activity on Center Response
a) Temporal frequency profile of OFF transient (left) and OFF sustained (right) ganglion cells in response
to a sinusoidally modulated 4.56◦ bar (50% contrast) with and without a 0.11 cyc/deg 50% contrast
square wave drifting grating in the background. We drifted the background grating at 2 Hz and 7.5
Hz and recorded the F1 amplitude of the center column’s response at different modulation frequencies.
Response amplitude is shown on top, and response phase is shown on bottom. b) Changing Vaa affects
the attenuation of center response from background stimulation. We recorded the F1 amplitude of OFF
transient (top) and OFF sustained (bottom) in response to a 4.56◦ bar whose intensity was modulated
at 5 Hz (50% contrast) while varying WA coupling, by changing Vaa . Responses are normalized to the
F1 response without background stimulation. Curves represent the F1 amplitude with a 2 Hz (triangle)
and 7.5 Hz (square) 0.11 cyc/deg 50% contrast square wave grating drifting in the far surround.
195
WA coupling is equivalent to increasing the resistance R of Equation 6.21, and we found
that, as expected, increasing the resistance caused loop gain to become more isolated at the
center location, and hence larger, which resulted in an attenuation of center response. For
small values of Vaa , there was no difference in ganglion center response with and without
a surround stimulation, since the center loop gain in this case is isolated from the rest of
the network and is therefore independent of surround stimulus. For large values of Vaa ,
however, the surround signal causes a further reduction in gain of the center, suggesting
that the surround stimulus, which causes surround loop gain, ws , to increase, was able to
more effectively communicate these changes to the center loop gain, wc , as described by
Equation 6.21. We expected to find a significant difference in attenuation between the 2 and
7.5 Hz surround grating, since the 7.5 Hz signal should induce a larger surround loop gain
ws . However, we only observed a larger attenuation in ganglion cell response with the 7.5
Hz surround stimulus for large valued of Vaa when we recorded OffT responses while OffS
responses demonstrated the same attenuation from both 2 and 7.5 Hz surround gratings.
7.4
Summary
Our retinomorphic chip recreates the cone pathway’s functionality qualitatively at the same
spatial scale, exhibiting luminance adaptation, bandpass spatiotemporal filtering, and contrast gain control. However, our chip consumes 17 µW per ganglion cell (62.7 mW total),
at an average spike rate of 45 spikes/second - one thousand times the 18 nW that a retinal
ganglion cell uses. Our estimate is based on a metabolic rate of 82 µmoles of ATP/g/min for
rabbit retina[1], divided among 300,000 ganglion cells. Ongoing advances in chip fabrication
technology will allow us to improve our chip’s energy efficiency as well as its spatial resolution and dynamic range. By using as little power-and weight and space-as the retina does,
196
retinomorphic chips could eventually serve as an in situ replacement, surpassing current
retinal prosthesis designs based on an external camera and processor[82]. More fundamentally, though, our results suggest that the we have replicated much of the complex processing
of the outer and inner retina neurocircuits by implementing these circuits in silicon. Our
chip realizes intensity adaptation, contrast gain control, and temporal adaptation by processing signals through complementary ON and OFF channels. We have extended earlier
retinomorphic designs[67, 11, 8] by including inner retina circuitry and by introducing a
novel push-pull architecture for this processing.
197
Chapter 8
Conclusion
This thesis has described our efforts to quantify some of the computations realized by
the mammalian retina in order to model this first stage of visual processing in silicon.
The retina, an outgrowth of the brain, is the most studied and best understood neural
system. A study of its seemingly simple architecture reveals several layers of complexity
that underly its ability to convey visual information to higher cortical structures. The retina
efficiently encodes this information by using multiple representations of the visual scene,
each communicating a specific feature found within that scene. Because of the complexity
inherent in the retina’s design, our strategy has been to develop a simplified model that
captures most of the relevant processing realized by the retina. Our model, and the silicon
implementation of that model, produces four parallel representations of the visual scene that
reproduce the retina’s four major output pathways and that incorporate fundamental retinal
processing and nonlinear adjustments of that processing, including luminance adaptation,
contrast gain control, and nonlinear spatial summation.
198
To quantify how the retina processes visual information, we recorded ganglion cell intracellular responses to a white noise stimulus. By using a white noise analysis, we were able to
represent retinal filtering with a simple model composed of a linear filter, a biphasic impulse
response that describes the temporal structure of the ganglion cell response, followed by a
rectified static nonlinearity that describes both the ganglion cell’s synaptic inputs and spike
generating mechanism. Although the solution to the parameters of this model is not unique,
such a model allows us to compare how these filters change across stimulus conditions, and
therefore how the retina adjusts its computations to adapt to different stimuli.
Our model for retinal processing uses a similar coding strategy to the one revealed by
our physiological measurements — input signals are bandpass filtered, in space and time,
and rectified into complementary pathways. However, through the white noise analysis, we
found that although ON and OFF ganglion cells were nearly identical, OFF cells exhibit a
stronger rectification in their nonlinearity, possibly reflecting differences in synaptic input
and baseline spike rates. Such differences may play an important role in encoding visual
information — an ON pathway may be more sensitive to smaller signals at the expense
of spatial resolution, while an OFF pathway may sacrifice sensitivity to maintain spatial
resolution. These differences complicate overall retinal structure, and we chose to ignore
these differences when constructing our retinal model, instead representing parallel ON and
OFF pathways as complementary and symmetric.
Through an overview of retinal anatomy and through preliminary physiological studies,
we were able to generate a general picture for how the retina processes visual information.
However, this picture is incomplete unless we take information theoretic considerations into
account. Maximizing information rates requires the retina to have an optimal filter that
whitens frequencies where signal power exceeds the noise, peaking at a cutoff determined
by stimulus and noise power, and that attenuates regions where noise power exceeds signal
199
power. This filter adjusts its temporal cutoff frequency based on the frequency of the input
by increasing the cutoff linearly with velocity to maintain maximal information rates. Our
simplified model for retinal structure realizes many of the features dictated by information
theory. A linear filtering scheme realized by the reciprocal interactions in the outer retina
recreates the optimal static filter predicted by information theory — the peak of this filter
lies at a fixed spatial frequency for low temporal frequencies and at a fixed temporal frequency for low spatial frequencies. In addition, modulation of narrow-field amacrine cell
presynaptic inhibition in our model for the inner retina allows the system to adapt linearly
to input frequency and communicate these changes laterally. We find that our inner retina
model adjusts its time constant to track the temporal frequency of the input, maintaining
this dynamic optimal filtering strategy.
In addition to adjustments the retina realizes in its spatiotemporal filter in response to
different input velocities, we know from previous work that the retina also adjusts its filters
in response to contrast. To verify this and to discriminate between central and peripheral
adaptive circuits, we returned to our white noise analysis to determine how these retinal
filters change across stimulus conditions. As expected, increases in central contrast cause a
gain reduction and a temporal speed up in the ganglion cell response, which is much more
pronounced in the spikes. These changes are instantaneous, suggesting that the longer time
course in gain changes, identified by previous studies as contrast adaptation, may be an
artefact if the non-uniqueness of the white noise solution. The change in the ganglion cell’s
temporal profile is even larger when increases in contrast are tuned to drive the excitatory
subunits, the bipolar cells, that converge to produce the ganglion cell’s center response. On
the other hand, increasing contrast in the surround causes a similar gain reduction, but has
no significant effect on the timing. This suggests the presence of two mechanisms by which
the retina adjusts its ganglion cell response to input stimuli. Our preliminary data suggests
that these two mechanisms can be discriminated using pharmacological techniques, although
200
future work to confirm the presence of and to explain the cellular mechanisms underlying
these separate mechanisms is needed.
We incorporated these adjustments in our model for processing in the inner retina.
The modulation of narrow-field amacrine cell presynaptic inhibition not only tracks input
frequency, but allows the system to adjust its temporal profile to input contrast. The
wide-field amacrine cell, which realizes this modulation, computes contrast and changes the
system’s closed-loop gain and time constant accordingly. Hence, our inner retina structure
realizes contrast gain control that is similar to our physiological observations. Furthermore,
our inner retina structure also realizes the adaptations induced by peripheral stimulation
— signals in the far surround cause a reduction in system gain in our model that does not
affect the system’s temporal profile.
Our simplified model for retinal structure hence realizes many of the features that define
visual processing in the mammalian retina. In our model for the outer retina, the interaction
between an excitatory cone network, which has relatively small space and time constants,
and an inhibitory horizontal cell network, which has larger space and time constants, creates a bandpass spatiotemporal response. This model adapts to input luminance through
horizontal cell modulation of cone-coupling and cone excitation, through autofeedback, to
produce a contrast signal at the cone terminal. Bipolar cells in our model rectify these
signals into complementary ON and OFF channels to replicate the parallel pathways of the
retina. In the inner retina, adjustments of the closed-loop system gain and time constant
by wide-field amacrine cell modulation of narrow-field amacrine cell presynaptic inhibition
realize contrast gain control and dynamic filtering. We morphed this model into CMOS
circuits by remaining faithful to the underlying biology — we connected neural primitives
that are based on the anatomy and physiology of the retina to generate a silicon circuit
that replicates most of the relevant processing found in the retina.
201
Testing our retinomorphic chip demonstrates that we have indeed replicated much of
the functionality of the retina’s cone pathway. We realized these computations by directly
studying the cellular interactions found in the retina and by implementing these interactions
using physiologically- and anatomically-based CMOS circuits. This chip provides a realtime model for the early stages of visual processing, based on the retina’s structure, at the
same spatial scale. Based on our estimates, our chip still uses roughly one thousand times
as much power as the mammalian retina, but we expect to address some of these power
issues in future designs. Coupling in the chip substrate between different components of
our retina circuit may cause unwanted currents to flow. Better isolating these components
in the layout design will hopefully eliminate much of this power consumption.
We also hope to further close the gap between chip performance and the performance
of the mammalian retina by redesigning some of the underlying circuitry in our CMOS
circuit. For example, asymmetric rectification in our model at the bipolar cells may cause
the differences in sensitivity we observe on separate channels. The mammalian retina also
exhibits an asymmetry at the bipolar cell, but in this case, the asymmetry lies in the
quiescent levels of activity in ON and OFF pathways. Our asymmetry forces an undesired
saturation in the ON channel that we could avoid by implementing a different bipolar
circuit. Additionally, chip transient ganglion cells exhibit contrast gain control while the
effect in chip sustained cells is significantly less pronounced. We hypothesize that this
difference arises from the presence or absence of feedforward amacrine cell inhibition. Thus,
an additional design issue to address would be to incorporate feedforward inhibition, that
could potentiate the effects of contrast, while maintaining the sustained behavior of our
narrow-field sustained-type ganglion cells.
While these design and power issues are important and still need to be addressed, our
chip has succeeded in replicating much of the relevant processing realized by the mam-
202
malian retina. Extensions of this work can be used to gain a deeper understanding of the
computations in the retina, to facilitate the design and fabrication of more complicated
neural systems in silicon, and for direct clinical applications. First, with a real-time model
of retinal processing that is easily adjusted, we can explore how certain components of our
model affect ganglion cell response. Second, neural systems that replicate processing in the
thalamus and in higher cortical structures rely on sensory input, and our retinomorphic
chip can serve as the front-end for these systems. Finally, by using as little power-and
weight and space-as the retina does, retinomorphic chips could eventually serve as an in
situ replacement, surpassing current retinal prosthesis designs based on an external camera
and processor. More fundamentally, though, our results suggest that the we can replicate
the complex processing found in neural circuits by implementing these circuits in silicon.
203
Appendix A
Physiological Methods
In order to quantify mammalian retina’s response behavior, we recorded intracellular membrane potentials from guinea pig retinal ganglion cells. We removed an eye from a guinea
pig anesthetized with ketamine/xylazine (1.0 cc kg−1 ) and pentobarbital(3.0 cc kg−1 ), following which the animal was killed by anesthetic overdose. We performed these procedures
in accordance with University of Pennsylvania and NIH guidelines. We mounted the whole
retina, including the pigment epithelium and choroid, flat in a chamber on a microscope
stage. We superfused the retina (∼5ml/min) with oxygenated (95% O, 5% CO2) Ames
medium1 at 34◦ . Acridine orange (0.001)%2 added to the superfusate allowed ganglion cell
somas to be identified by fluorescence during brief exposure to near UV light. We targeted
large somas (20-25µm) in the visual streak for intracellular recording. Glass electrodes
(tip resistance 80-200 MΩ) contained 1% pyranine3 and 2% Neurobiotin4 in 2M potassium
1
Sigma, St. Louis, MO
Molecular Probes, Eugene, OR
3
Molecular Probes
4
Vector Laboratories, Burlingame, CA
2
204
acetate.
Membrane potential was amplified5 , continuously sampled at 5kHz and stored on computer6 . We analyzed data with programs written in Matlab7 . Spikes were detected off-line
and removed computationally to allow analysis of membrane potential[30]. We determined
the resting potential by averaging membrane potential over two seconds before and after
each stimulus. We subtracted the resting potential from all recordings to analyze intracellular deviations from rest.
We displayed input stimuli on a miniature computer monitor8 projected through the
top port of the microscope through a 2.5X objective and focused on the photoreceptors.
Mean luminance of the green phosphor corresponded to ∼105 isomerizations cone−1 sec−1 .
Monitor resolution was 825 × 480 pixels with 60Hz vertical refresh; stimuli were confined
to a square with 430 pixels to a side (3.7mm on the retina). A typical receptive field center
was ∼75 pixels diameter. The relationship between gun voltage and monitor intensity
was linearized in software with a lookup table. We programmed stimuli in Matlab using
extensions provided by the high-level Psychophysics Toolbox[14] and the low-level Video
Toolbox[79].
5
NeuroData, IR-283, NeuroData Instruments Corp., Delaware Water Gap, PA
AxoScope, Axon Instruments, Foster City, CA
7
Mathworks, Natick, MA
8
Lucivid MR1-103, Microbrightfield, Colchester, VT
6
205
Bibliography
[1] A. Ames, Y.Y. Li, E.C. Heher, and C.R. Kimble. Energy metabolism of rabbit retina
as related to function: high cost of Na+ transport. J Neurosci, 12:840–853, 1992.
[2] J. Atick and N. Redlich. What does the retina know about natural scenes. Neural
Computation, 4(2):196–210, 1992.
[3] G.B. Awatramani and M. Slaughter. Intensity-dependent, rapid activation of presynaptic metabotropic glutamate receptors at a central synapse. J Neurosci, 21:741–749,
2001.
[4] H.B. Barlow. Summation and inhibition in the frog’s retina. J Physiol, 119:69–88,
1953.
[5] H.B. Barlow. Possible principles underlying the transformations of sensory messages.
In WA Rosenblith, editor, Sensory Communication, pages 217–234. MIT Press, Cambridge, MA, 1961.
[6] E.A. Benardete and E. Kaplan. The receptive field of the primate P retinal ganglion
cells. Vis Neurosci, 14:187–205, 1997.
[7] M.J. Berry and M. Meister. Refractoriness and neural precision. J Neurosci, 18:2200–
2211, 1998.
206
[8] K. Boahen. A retinomorphic chip with parallel processing: Encoding increasing, on,
decreasing, and off visual signals. In Press.
[9] K.A. Boahen. A throuput-on-demand address-event transmitter for neuromorphic
chips. In J.E. Moody, editor, Advances in neural information processing 4, volume 4.
San Mateo, CA, 1991.
[10] K.A. Boahen. Retinomorphic Vision Systems: Reverse Engineering the Vertebrate
Retina. PhD thesis, California Institute of Technology, Pasadena, CA, 1997.
[11] K.A. Boahen and A. Andreou. A contrast-sensitive retina with reciprocal synapses.
In Conference on Advanced Research in VLSI, Los Alamos, CA, 1999.
[12] B.B. Boycott and H. Wassle. The morphological types of ganglion cells of the domestic
cat’s retina. J Physiol, 240:297–419, 1974.
[13] B.B. Boycott and H. Wassle. Morphological classification of bipolar cells of the primate
retina. J Neurosci, 3:1069–1088, 1991.
[14] D.H. Brainard. The psychophysics toolbox. Spat Vis, 10:433–436, 1997.
[15] D. Calkins and P. Sterling. Absence of spectrally specific lateral inputs to midget
ganglion cells in primate retina. Nature, 381:613–615, 1996.
[16] D. Calkins, Y. Tsukamoto, and P. Sterling. Foveal cones form basal as well as invaginating contacts with diffuse on bipolar cells. Vision Res, 36:3373–3381, 1996.
[17] E.J. Chichilnisky. A simple white noise analysis of neuronal light responses. Network:
Comput Neural Sys, 12:199–213, 2001.
[18] E.J. Chichilnisky and R.S. Kalmar. Functional asymmetries in on and off ganglion
cells of primate retina. J Neurosci, 2001.
207
[19] M.H. Chun and H. Wassle. Gaba-lime immunoreactivity in the cat retina: electron
microscopy. J Comp Neurol, 279:55–67, 1989.
[20] Passaglia C.L., Enroth-Cugell C., and Troy J.B. Effects of remote stimulation on the
mean firing rate of cat retinal ganglion cells. J Neurosci, 21:5794–5803, 2001.
[21] B.G. Cleland and W.R. Levick. Brisk and sluggish concentrically organized ganglion
cells in the cat’s retina. J Physiol, 240:421–456, 1974.
[22] E. Cohen and P. Sterling. Convergence and divergence of cones onto bipolar cells in
the central area of cat retina. Phil Trans R Soc Lond B, 330:305–321, 1990.
[23] E. Cohen and P. Sterling. Parallel circuits from cones to the on-beta ganglion cell.
Eur J Neurosci, 4:506–520, 1992.
[24] L.J. Croner and E. Kaplan. Receptive fields of P and M ganglion cells across the
primate retina. Vision Res, 35(1):7–24, 1995.
[25] C.A. Curcio, K.R. Sloan, R.E. Kalina, and A.E. Hendrickson. Human photoreceptor
topography. J Comp Neurol, 292:497–523, 1990.
[26] D.M. Dacey. Axon-bearing amacrine cells of the macaque monkey retina. J Comp
Neurol, 284:275–293, 1989.
[27] R.F. Dacheaux and E. Raviola. The rod pathway in the rabbit retina: a depolarizing
bipolar and amacrine cell. J Neurosci, 6:331–345, 1986.
[28] R.F. Dacheaux and E. Raviola. Light responses from one type of ON-OFF amacrine
cell in the rabbit retina. J Neurophysiol, 74:2460–2467, 1995.
[29] F.M. de Monasterio, S.J. Schein, and E.P. McCrane. Staining of blue-sensitive cones
of the macaque retina by a fluorescent dye. Science, 213:1278–1281, 1981.
208
[30] J.B. Demb, L. Haarsma, M. Freed, and P. Sterling. Functional circuitry of the retinal
ganglion cell’s nonlinear receptive field. J Neurosci, 19:9756–9767, 1999.
[31] J.B. Demb, K.A. Zaghloul, L. Haarsma, , and P. Sterling. Bipolar cells contribute to
nonlinear spatial summation in the brisk-transient (Y) ganglion cell in mammalian
retina. J Neurosci, 21:7447–7454, 2001.
[32] D.W. Dong and J.J. Atick. Statistics of natural time-varying images. Network, 6:345–
358, 1995.
[33] R. Douglas, M. Mahowald, and C. Mead. Neuromorphic analogue VLSI. Annu Rev
Neurosci, 18:255–281, 1995.
[34] J.E. Dowling. Information processing by local circuits: The vertebrate retina as a
model system. In F.O. Schmitt and F.O. Worden, editors, The Neurosciences: Fourth
Study Program. MIT Press, Cambridge, MA, 1979.
[35] J.E. Dowling. Dopamine: a retinal neuromodulator? Trends Neurosci, 9:236–240,
1986.
[36] J.E. Dowling and B.B. Boycott. Organization of the primate retina: electron microscopy. Proc R Soc Lond B, 166:80–111, 1966.
[37] J.M. Enoch. Retinal receptor orientation and photoreceptor optics. In J.M. Enoch
and F.L.J. Tobey, editors, Vertebrate Photoreceptor Optics. Springer-Verlag, Berlin,
1981.
[38] C. Enroth-Cugell and A.W. Freeman. The receptive-field spatial structure of cat
retinal Y cells. J Physiol, 384:49–79, 1987.
[39] C. Enroth-Cugell and H.G. Jakiela. Suppression of cat retinal ganglion cell responses
by moving patterns. J Physiol, 302:49–72, 1980.
209
[40] C. Enroth-Cugell and J.G. Robson. The contrast sensitivity of retinal ganglion cells.
J Physiol, 187:512–552, 1966.
[41] C. Enroth-Cugell, J.G. Robson, D.E. Schweitzer-Tong, and A.B. Watson. Spatiotemporal interactions in cat retinal ganglion cells showing linear spatial summation.
J Physiol, 341:279–307, 1983.
[42] T. Euler, H. Schneider, and H. Wassle. Glutamate responses of bipolar cells in a slice
preparation of the cat retina. J Neurosci, 16:2934–2944, 1996.
[43] A.L. Fairhall, G.D. Lewen, W. Bialek, and R.R. de Ruyter can Steveninck. Efficiency
and ambiguity in an adaptive neural code. Nature, 412:787–792, 2001.
[44] E.V.Jr. Famiglietti and H. Kolb. Structural basis of On- and Off-center responses in
retinal ganglion cells. Science, 194:193–195, 1976.
[45] M. Freed, R. Pflug, H. Kolb, and R. Nelson. ON-OFF amacrine cells in cat retina. J
Comp Neurol, 364:556–566, 1996.
[46] M.A. Freed and P. Sterling. The On-alpha ganglion cell of the cat retina and its
presynaptic cell types. J Neurosci, 8:2956–2966, 1988.
[47] L.J. Frishman, A.W. Freeman, J.B. Troy, D.E. Schweitzer-Tong, and C. EnrothCugell. Spatiotemporal frequency responses of cat retinal ganglion cells. J Gen
Physiol, 89:599–628, 1987.
[48] G.D. Guiloff, J. Jones, and H. Kolb. Organization of the inner plexiform layer of the
turtle. J Comp Neurol, 272:280–292, 1988.
[49] S. Hochstein and R.M. Shapley. Linear and nonlinear spatial subunits in Y cat retinal
ganglion cells. J Physiol, 262:265–284, 1976.
210
[50] K.M. Hynna and K. Boahen. Space rate coding in an adaptive silicon neuron. Neural
Networks, 14:645–656, 2001.
[51] G.H. Jacobs. The distribution and nature of colour vision among the mammals. Biol
Rev, 68:413–471, 1993.
[52] R.J. Jensen and N.W. Daw. Effects of dopamine and its agonists and antagonists
on the receptive field properties of ganglion cells in the rabbit retina. Neuroscience,
17(3):837–855, 1986.
[53] M. Kamermans and F. Werblin. Gaba-mediated positive autofeedback loop controls
horizontal cell kinetics in tiger salamander retina. J Neurosci, 12(7):2451–63, 1992.
[54] E. Kaplan, B.B. Lee, and R.M. Shapley. New views of primate retinal function. In
N. Osborne and J. Chader, editors, Progress in Retinal Research. Pergamon Press,
Oxford, UK, 1990.
[55] D.H. Kelly. Motion and vision ii: Stabilized spatio-temporal threshold surface. J Opt
Soc Am, 69:1340–1349, 1979.
[56] J. Kim and F. Rieke. Temporal contrast adaptation in the input and output signals
of salamander retinal ganglion cells. J Neurosci, 21:287–299, 2001.
[57] H. Kolb. Organization of the outer plexiform layer of the primate retina: electron
microscopy of golgi-impregnated cells. Phil Trans R. Soc Lond B, 258:261–283, 1970.
[58] H. Kolb. The architecture of functional neurocircuits in the vertebrate retina. Invest
Opthalmol Vis Sci, 35:2385–2404, 1994.
[59] H. Kolb, N. Cuenca, H.H. Wang, and L. Dekorver. The synaptic organization of the
dopaminergic amacrine cell in the cat retina. J Neurocytol, 19:343–366, 1990.
211
[60] H. Kolb and R. Nelson. Functional neurocircuitry of amacrine cells in the cat retina.
In A. Gallego and P. Gouras, editors, Neurocircuitry of the retina: A Cajal memorial.
Elsevier, New York, NY, 1985.
[61] H. Kolb and R. Nelson. Off-alpha and Off-beta ganglion cells in cat retina: Ii. neural
circuitry as revealed by electron microscopy of hrp stains. J Comp Neurol, 329(1):85–
110, 1993.
[62] H. Kolb, R. Nelson, and A. Mariani. Amacrine cells, bipolar cells and ganglion cells
of the cat retina: a golgi study. Vision Res, 21:1081–1114, 1981.
[63] M.J. Korenberg and I.W. Hunter. The identification of nonlinear biological systems:
Lnl cascade models. Biol Cybern, 55:125–134, 1986.
[64] S.W. Kuffler. Discharge patterns and functional organization of mammalian retina.
J Neurophysiol, 16:37–68, 1953.
[65] S. Laughlin, R.R. de Ruyter van Steveninck, and J.C. Anderson. The metabolic cost
of neural information. Nature Neuroscience, 1(1):36–41, 1998.
[66] G. Maguire and P. Lukasiewicz. Amacrine cell interactions underlying the response
to change in tiger salamander retina. J Neurosci, 9:726–35, 1989.
[67] M. Mahowald and C. Mead. A silicon model of early visual processing. Neural
Networks, 1, 1988.
[68] P. Marmarelis and V. Marmarelis. Analysis of Physiological Systems: The White
Noise Approach. Plenum Press, New York, NY, 1978.
[69] P.Z. Marmarelis and K.I. Naka. White noise analysis of a neuron chain: an application
of the wiener theory. Science, 175:1276–1278, 1972.
212
[70] R.H. Masland, J.W. Mills, and C. Cassidy. The functions of acetylcholine in the rabbit
retina. Proc R Soc Lond B, 223:121–139, 1984.
[71] S.C. Massey and G. Maguire. The role of glutamate in retina circuitry. In Excitatory
Amino Acids and Synaptic Transmission. Academic Press Ltd, 1995.
[72] S.C. Massey and D.A. Redburn. Light evoked release of acetylcholine in response
to a single flash: cholinergic amacrine cells receive on and off input. Brain Res,
328:374–377, 1985.
[73] G. Matthews. Neurotransmitter release. Ann Rev Neurosci, 19:219–233, 1996.
[74] C. Mead. Analog VLSI and Neural Systems. Addison Wesley, Reading, MA, 1989.
[75] M. Meister and M.J. Berry. The neural code of the retina. Neuron, 22:435–450, 1999.
[76] M. Murakami, E.I. Miyachi, and K.I. Takahashi. Modulation of gap junctions between
horizontal cells by second messengers. In Progess in Retinal and Eye Research. Elsevier
Science Ltd, London, 1995.
[77] R. Nelson, R.V.Jr. Famiglietti, and H. Kolb. Intracellular staining reveals the different
levels of stratification for on- and off-center ganglion cells in cat retina. J Neurophysiol,
41:472–483, 1978.
[78] O. Packer, A. Hendrickson, and C. Curcio. Photoreceptor topography of the retina in
the adult pigtail macaque (macaca nemestrina). J Comp Neurol, 288:165–183, 1989.
[79] D.G. Pelli. The video toolbox software for visual psychophysics: transforming numbers
into movies. Spat Vis, 10:437–442, 1997.
[80] E.N. Pugh and T.D. Lamb. Amplification and kinetics of the activation steps in
phototransduction. Biochim Biophys Acta, 1141:111–149, 1993.
213
[81] R. Rao-Mirotznik, A. Harkins, G. Buchsbaum, and P. Sterling. Mammalian rod
terminal: Architecture of a binary synapse. Neuron, 14:561–569, 1995.
[82] J.F. Rizzo, J. Wyatt, M. Humayun, E. de Juan, W. Liu, A. Chow, R. Eckmiller,
E. Zrenner, T. Yagi, and G. Abrams. Retinal prosthesis: An encouraging first decade
with major challenges ahead. Opthalmology, 108(1):13–4, 2001.
[83] R.W. Rodieck. Quantitative analysis of cat retinal ganglion cell response to visual
stimuli. Vision Res, 5:583–601, 1965.
[84] R.W. Rodieck. The primate retina. Comp Primate Biol, 4:203–278, 1988.
[85] R.W. Rodieck and R.K. Brening. Retinal ganglion cells: Properties, types, genera,
pathways, and trans-species comparisons. Brain Behav Evol, 23:121–164, 1983.
[86] B. Roska and F. Werblin. Vertical interactions across ten parallel, stacked representations in the mammalian retina. Nature, 410:583–587, 2001.
[87] H.A. Saito. Morphology of physiologically identified X-, Y-, and W-type retinal ganglion cells of the cat. J Comp Neurol, 221:279–288, 1983.
[88] H.M. Sakai and K. Naka. Response dynamics and receptive-field organization of
catfish ganglion cells. J Gen Physiol, 105:795–814, 1995.
[89] H.M. Sakai, K. Naka, and M.J. Korenberg. White-noise analysis in visual neuroscience. Visual Neuroscience, 1:287–296, 1988.
[90] H.M. Sakai and K.I. Naka. Novel pathway connecting the outer and inner vertebrate
retina. Nature, 315:570–571, 1985.
[91] D. Sandmann, B.B. Boycott, and L. Peichl. The horizontal cells of artiodactyl retina:
a comparison with Cajal’s description. Vis Neurosci, 13:735–746, 1996.
214
[92] C. Shannon and W. Weaver. A Mathematicl Theory of Communication. University
of Ilinois Press, Chicago, IL, 1948.
[93] R.M. Shapley and J.D. Victor. Nonlinear spatial summation and the contrast gain
control of cat retinal ganglion cells. J Physiol, 290:141–161, 1979.
[94] R.M. Shapley and J.D. Victor. How the contrast gain control modifies the frequency
responses of cat retinal ganglion cells. J Physiol, 318:161–179, 1981.
[95] S.M. Smirnakis, M.J. Berry, D.K. Warland, W. Bialek, and M. Mesiter. Retinal
processing adapts to image contrast and spatial scale. Nature, 386:69–73, 1997.
[96] R.G. Smith. Simulation of an anatomically defined local circuit - The cone-horizontal
cell network in cat retina. Visual Neurosci, 12:545–561, 1995.
[97] R.G. Smith, M.A. Freed, and P. Sterling. Microcircuitry of the dark-adapted cat
retina: functional architecture of the rod-cone network. J Neurosci, 6:3505–3517,
1986.
[98] P. Sterling. Microcircuitry of the cat retina. Ann Rev Neurosci, 6:149–185, 1983.
[99] P. Sterling. Retina. In G.M. Shepherd, editor, The Synaptic Organization of the
Brain. Oxford University Press, New York, NY, fourth edition, 1998.
[100] P. Sterling, M.A. Freed, and R.G. Smith. Architecture of the rod and cone circuits
to the On-beta ganglion cell. J Neurosci, 8:623–642, 1988.
[101] J. Stone and Y. Fukuda. Properties of cat retinal ganglion cells: a comparison of
W-cells with X- and Y-cells. J Neurophysiol, 37:722–748, 1974.
[102] J.B. Troy and C. Enroth-Cugell. X and Y ganglion cells inform the cat’s brain about
contrast in the retinal image. Exp Brain Res, 93:383–390, 1993.
215
[103] J.H. van Hateren. Real and optimal neural images in early vision. Nature, 360:68–70,
1992.
[104] J.H. van Hateren. A theory of maximizing sensory information. Biol Cyb, 68:23–29,
1992.
[105] D. Vaney. The mosaic of amacrine cells in mammalian retina. In N. Osborne and
J. Chader, editors, Progess in Retinal Research. Pergamon Press, Oxford, UK, 1990.
[106] D.I. Vaney. Patterns of neuronal coupling in the retina. Prog Ret & Eye Res, 13:301–
355, 1994.
[107] J.D. Victor. The dynamics of cat retinal X cell centre. J Physiol, 386:219–246, 1987.
[108] F.S. Werblin and J.E. Dowling. Organization of the retina of the mudpuppy, necturus
maculosus ii. intracellular recording. J Neurophysiol, 32:339–355, 1969.
[109] D. Williams, N. Sekiguchi, and D. Brainard. Color, contrast sensitivity, and the cone
mosaic. Proc Natl Acad Sci, 21:9770–9777, 1993.
[110] H.M. Young and D.I. Vaney. Rod-signal interneurons in the rabbit retina: 1. rod
bipolar cells. J Comp Neurol, 310:139–153, 1991.
216
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement