Predicting Every Spike: A Model for the Responses of Visual Neurons

Predicting Every Spike: A Model for the Responses of Visual Neurons
Neuron, Vol. 30, 803–817, June, 2001, Copyright 2001 by Cell Press
Predicting Every Spike: A Model
for the Responses of Visual Neurons
Justin Keat,1 Pamela Reinagel,2 R. Clay Reid,2
and Markus Meister1,3
Molecular and Cellular Biology
Harvard University
16 Divinity Avenue
Cambridge, Massachusetts 02138
Harvard Medical School
Boston, Massachusetts 02115
In the early visual system, neuronal responses can be
extremely precise. Under a wide range of stimuli, cells
in the retina and thalamus fire spikes very reproducibly, often with millisecond precision on subsequent
stimulus repeats. Here we develop a mathematical
description of the firing process that, given the recent
visual input, accurately predicts the timing of individual spikes. The formalism is successful in matching the
spike trains from retinal ganglion cells in salamander,
rabbit, and cat, as well as from lateral geniculate nucleus neurons in cat. It adapts to many different response types, from very precise to highly variable. The
accuracy of the model allows a compact description
of how these neurons encode the visual stimulus.
A central problem of research in neural coding is the
relationship between neural activity in the brain and the
behaviorally relevant variables of the external world. In
the sensory periphery, those variables are the inputs
from sense organs. In the visual system specifically, they
are completely defined by the light intensity projected on
the retina, as a function of wavelength, space, and time.
For a given neuron in the visual system, one would like
to have a compact description that predicts the spike
train in response to any such visual stimulus. If such a
neural code could be found, it would specify what aspects of the world this neuron represents and what it
could communicate to other neurons in the circuit. Furthermore, the code would specify what aspects of the
spike train are used to represent those features. As a
result, one could better understand the neuron’s role in
the information processing task of the entire circuit.
One obstacle to a simple mathematical description
of sensory responses is that neurons tend to produce
variable spike trains even to identical repeats of the
same stimulus. For any given stimulus, there are many
possible responses, and a complete neural code would
specify the probability that each of these will occur.
Obviously, a probability table relating every possible
stimulus to every possible spike train would be somewhat unwieldy, and therefore some simplifying assumptions are in order. A popular assumption is that the
Correspondence: [email protected]
neuron generates spikes independently of each other:
at every instant it “decides” whether or not to fire a
spike, and the probability of firing varies with time as a
function of the stimulus. In this limit, the probabilities of
getting various spike trains are completely determined
by the time course of the instantaneous firing rate r(t)
(Rieke et al., 1997; Meister and Berry, 1999). Experimentally, this can be estimated by accumulating a peristimulus time histogram (PSTH) of spikes from many identical
trials. The problem of neural coding is then reduced to
predicting the firing rate r(t) as a functional of the sensory
stimulus s(t) (Rodieck, 1965; Sakuranaga and Naka,
1985; Victor, 1987).
The assumption that sensory responses are fully described by the time-varying firing rate may well be justified in certain brain areas, for example, deep in the
cortex (Shadlen and Newsome, 1994). Neurons there
generate highly variable spike trains because they receive large numbers of unsynchronized synaptic inputs
and because many of these are not under experimental
control. By contrast, neurons in the early visual system—
from the retina to the lateral geniculate nucleus (LGN)
to area V1—can deliver remarkably reproducible spike
trains, whose trial-to-trial variability is clearly lower than
predicted from the simple firing rate formalism (Berry et
al., 1997; Reich et al., 1997; Berry and Meister, 1998;
Kara et al., 2000; Reinagel and Reid, 2000). Often, individual action potentials are reproducibly time locked to
the stimulus. Given that the synapses in this pathway
leading to visual cortex are especially reliable (Reid and
Alonso, 1995; Usrey et al., 1999), individual spikes can,
in fact, have a strong effect on postsynaptic neurons.
Consequently, one needs a different framework for
studying this neural code that allows the prediction of
individual spikes and spike patterns with high timing
accuracy, but also accounts for the remaining stochastic
variability in these responses.
When exposed to a visual stimulus rich in temporal
variation—such as natural movies or random flicker—
retinal ganglion cells typically respond with brief clusters
of spikes separated by longer periods of silence (Berry
et al., 1997; Berry and Meister, 1998; Meister and Berry,
1999) (Figure 1A). If the stimulus is repeated, the clusters
occur reliably at the same times, sometimes to within 1
ms. Thus, the PSTH accumulated over many trials is a
series of sharp peaks of high firing rate separated by
intervals where the firing rate is absolutely zero (Figure
1B). Similar sharp firing events in response to temporally
rich stimuli have been reported in relay cells of the lateral
geniculate nucleus (Dan et al., 1996; Reinagel and Reid,
2000) in the nucleus of the optic tract (Clifford and Ibbotson, 2000), in area V1 of the primate visual cortex (Vinje
and Gallant, 2000), and even in cortical area MT (Bair
and Koch, 1996; Buracas et al., 1998).
The spike clusters produced by retinal ganglion cells
are reproducible not only in their timing but also in the
number of spikes they contain: the trial-to-trial variance
in the spike count of a particular firing event is often
less than one and almost always less than the mean
spike count (Berry et al., 1997; Berry and Meister, 1998;
match the real ones. In almost all cases, the algorithm
makes successful predictions: the simulated spike
trains are about as close to the real spike trains as the
real spike trains are across trials. Remarkably, the model
can capture the behavior of a wide range of cell types,
with seemingly very different light responses.
Figure 1. Precise Firing Events in the Response of a Retinal Ganglion Cell
(A) The intensity time course of the flicker stimulus (top trace) and
responses of a salamander “strong OFF” cell to 12 repetitions of
that stimulus. Each vertical mark represents a spike.
(B) Parsing a set of spike trains into events (see Experimental Procedures for details). The PSTH accumulated across trials (thin curve)
is smoothed (thick curve) with a Gaussian filter. When two peaks in
the smoothed curve are separated by a significant trough, a division
(vertical gray lines) is drawn between the two firing events.
(C) Matching real (top) and predicted (bottom) firing events. Each
data point represents event time T and spike number N of the corresponding event. The error bars indicate the variability across trials
V of the event time and the variability S of the spike number. Arrows
show the correspondences that minimize the error measure. Dotted
arrows illustrate an illegal “crossed” correspondence (see Experimental Procedures).
Kara et al., 2000). Thus, spike clusters differing by just
one spike can reliably convey different messages about
the visual stimulus, and they do so with a timing precision of a few milliseconds. The key variables by which
a firing event conveys its visual message are the time
of the first spike and the total number of spikes. By
comparison, the detailed timing of subsequent spikes
within the firing event contributes little to neural coding,
carrying only ⵑ5% of the visual information (M. Berry
and M.M., unpublished data). However, these prior studies did not address just what features of the stimulus
these firing events convey.
Here we develop a mathematical model that can predict neuronal spike trains from the time course of the
stimulus. The model is applied to recordings of responses to random flicker stimuli from retinal ganglion
cells in salamander, rabbit, and cat, as well as LGN
neurons in cat. For each neuron, we optimize the parameters of the model so that the simulated responses
As discussed above, rapidly varying stimuli elicit precise
and reliable responses from many visual neurons. For
example, Figure 1 illustrates the response of a salamander retinal ganglion cell to randomly flickering light. The
neuron fires at quite precisely defined moments during
the flicker sequence (Figure 1A). Each firing event consists of a small cluster of spikes, and subsequent events
are clearly separated by periods of complete silence
(Figure 1B). Our mathematical model of this process
should take the visual stimulus as input and generate
spike trains with discrete firing events that closely resemble those of the real neuron. To evaluate this correspondence quantitatively, we measure several salient
properties of these firing events in real and simulated
spike trains.
Following previous work (Berry et al., 1997; Berry and
Meister, 1998), the most important aspects of a firing
event are its time of occurrence T, measured as the trialaveraged time of the first spike, and its spike number
N, measured as the trial-averaged number of spikes in
the event. Though the spike trains appear reproducible,
there clearly is some trial-to-trial variability (Figure 1B).
Therefore, we also measure the time jitter V as the standard deviation across trials of the time of the first spike,
and the number jitter S as the standard deviation across
trials of the spike number. In this way, each firing event
is characterized by four numbers (T, N, V, S) (Figure 1C;
Equation 9 [all equations in Experimental Procedures,
below]). The model’s goal is to produce simulated spike
trains that match these event properties as closely as
possible. To this end, the model’s parameters are adjusted for the optimal fit during a flicker sequence that
typically elicits tens to hundreds of firing events. Then
the model’s predictions are evaluated on the neuronal
responses to a different stimulus sequence. Details of
the fitting and testing process are presented in Experimental Procedures. In the following sections, the model
is assembled stepwise, and at each stage of improvement, we illustrate its successes and remaining deficits
by comparing representative real spike trains with the
predictions of the model.
Predicting the Occurrence of an Event
We begin with a simple algorithm to predict the times
T when firing events occur, without regard to the number
of spikes they contain or the variability across trials
(Figure 2A). The stimulus s(t) represents the light intensity as a function of time. This is passed through a linear
filter to produce the generator potential g(t). When g(t)
crosses a preset threshold from below, an event is fired.
When the function passes back through threshold from
above, nothing happens.
The only parameters of this model are the filter, given
by the impulse response F(t), and the threshold ␪ [this
Modeling Precise Spike Trains from Visual Neurons
Figure 3. Predicting All Spikes
Figure 2. Predicting Only the Times of Firing Events
(A) The stimulus s(t) is convolved with a filter F(t) to produce the
generator potential g(t). An event is fired when the generator potential crosses upward through the threshold ␪.
(B) A brief segment of the fit to responses from a salamander “strong
OFF” retinal ganglion cell. r(t) shows the times of the first spike in
each event, recorded during a single trial. r⬘(t) shows the predicted
event times when g(t) (solid line) crosses the threshold (dashed line).
(C) A histogram of inter-event intervals for a real cell (thick line) and
the model’s fit (thin line), showing that the model predicts too many
events that closely follow other events.
version corresponds to a(t) ⫽ b(t) ⫽ P(t) ⫽ 0 in Equations
1–4]. Once the filter and threshold are optimized, the
correspondence between the real firing events and the
predicted ones is for the most part quite strong (Figure
2B). However, the model tends to overpredict the number of events that closely follow a preceding event, such
as the last event predicted in Figure 2B. Correspondingly, the number of events separated from the preceding one by a long interval is underpredicted (Figure 2C).
It appears that the model could be improved by implementing some partial refractoriness after a firing event.
Predicting All Spikes in an Event
The above scheme can be expanded to generate all of
the spikes within firing events by simply adding a negative feedback loop. In the model of Figure 3A, each spike
triggers a negative after-potential P(t) that gets added
to the generator potential g(t). Thus, the sum h(t) immediately drops below threshold, and if g(t) continues to rise,
the model will fire again. Thus, large excursions of g(t)
now lead to clusters of several spikes during the rising
phase (Figure 3B).
After such a firing event, h(t) is considerably lower
than g(t) as a result of the accumulated after-potentials,
(A) The spiking model of Figure 2A (gray) augmented by an additional
feedback pathway (black). The exponential feedback potential P(t)
is triggered by each spike and lowers h(t), the input to the threshold
(B) The fit to responses from a salamander “strong OFF” cell. r(t)
shows real spikes from a single trial, and r⬘(t) the predicted spikes.
The expanded time scale on the right shows details of repetitive
firing as a result of the feedback potential. The negative feedback
also suppresses events that closely follow other events (arrow).
(C) Histogram of inter-event intervals for the real cell (thick line) and
the model’s fit (thin line).
and thus the probability for subsequent firing events is
temporarily reduced until the after-potentials decay. For
example, the false event predicted in Figure 2B no longer
occurs in Figure 3B (arrow). The suppression of these
events allows the fitting algorithm to lower the model’s
spike threshold, which relieves the underprediction of
events separated by large intervals. Figure 3C shows
that most of the errors in Figure 2C are eliminated by
this addition. Thus, the negative feedback mechanism
serves both to simulate repetitive firing within a firing
event and to implement the refractoriness following an
Predicting the Variation across Trials
The last task is to predict the variability of the firing
events, specifically the trial-to-trial variations of the
event time, V, and of the spike number, S. We did so
by adding two Gaussian noise sources, as shown in
Figure 4A. Since the noise signals are different from trial
to trial, the simulated spike trains now vary across trials
as well (Figure 4B).
The first noise source, a(t), is added to the generator
potential prior to the threshold. This introduces random
variation in the time of the threshold crossing at the
Figure 4. Predicting Variability across Trials
(A) The spiking model of Figure 3A (gray) augmented by two noise sources (black).
(B) The fit to responses from a salamander
“strong OFF” cell. r(t) shows real spikes from
three trials, and r⬘(t) the predicted spikes on
three trials. Each simulated trial used different
choices for the noise waveforms a(t) and b(t),
and thus produced slightly different h(t), the
input to the threshold operation.
(C) The spike number variance S2 of firing
events plotted against their mean spike number N for a real retinal ganglion cell (left), for
the best-fit model using only a(t) (middle), and
for the best-fit model using both a(t) and b(t)
(right). The arched patterns result from the
fact that the spike number is necessarily integer, which constrains the possible values for
its variance (Berry et al., 1997). The dashed
line corresponds to the identity, the relationship expected for a rate-modulated Poisson
spike train.
beginning of a firing event. We chose a(t) to have a
Gaussian amplitude distribution with standard deviation
␴a and an exponentially decaying autocorrelation function with time constant ␶a. The time constant was fixed
at ␶a ⫽ 0.2 s for neurons from salamander and rabbit
retina, and ␶a ⫽ 0.02 s for cat neurons. These values
served very well in predicting the trial-to-trial jitter of
the onset of firing events, and holding them fixed helped
to reduce the number of free parameters of the model.
In the spirit of maintaining simplicity, we had hoped
that this single stochastic component would explain the
variability in both event timing and spike number. However, we found that the second and subsequent spikes
in a firing event are much more variable than expected
from a(t) alone. A model including only a(t) predicts a
very low trial-to-trial variation in the spike number of
firing events, considerably smaller than that observed
in real neurons (Figure 4C). This is because the optimal
value for the correlation time ␶a significantly exceeds
the duration of most firing events, and thus generation
of spikes after the first threshold crossing becomes essentially deterministic. The greater variability of real
spike trains could be explained if each spike injects
additional noise into the process. We implemented this
by invoking a second stochastic component b(t), which
randomly modulates the amplitude of the feedback potential P(t) following each spike. This noise source was
taken to have a Gaussian distribution with standard deviation ␴b and a very short correlation time, so its values
are independent from spike to spike. With this addition,
the model successfully accounts for the spike number
variability (Figure 4C).
Simulated Spike Trains
This model was used to fit the responses of neurons in
the retina and lateral geniculate nucleus. For each neuron, we identified the physiological cell type by traditional criteria (see Experimental Procedures), recorded
the response to a random flicker stimulus, and then
optimized the parameters in the mechanism of Figure
4A to match those spike trains. To illustrate the performance of this algorithm, we present raster plots of real
spike trains on several identical stimulus trials, along
with the corresponding predicted spike trains from the
best-fit model (Figure 5). These brief episodes of the
response were chosen to have prediction errors typical
for their respective cell type. In addition, an analysis of
all the firing events produced by one cell is given in
Figures 6 and 7, and Table 1 gives summary measures
of the model’s performance across all the neurons we
Modeling Precise Spike Trains from Visual Neurons
Figure 5. Comparison of Real and Simulated Spike Trains
Each raster plot is for a single cell and shows neural spike trains from several stimulus trials (top half) and corresponding spike trains predicted
by the model (bottom half). Graphs on the right show detail on an expanded time axis. The sample cells are from the following types:
salamander retina strong OFF (A), salamander retina ON (B), rabbit retina OFF brisk transient (C), rabbit retina delayed OFF (D), cat retina Y
ON (E), and cat LGN X ON (F). Note the different time scales: tick intervals are 1 s in left hand panels, 0.2 s in right hand panels.
analyzed. In interpreting these results, recall that the
model parameters were always derived from a different
stimulus segment than the one used for evaluating the
fits; in this sense, the simulated spike trains are truly
Figure 5A shows responses from a salamander
“strong OFF” ganglion cell. These neurons produce very
sparse spike trains: the firing rate, averaged over all
stimulus repeats, is exactly zero more than 94% of the
time because the spikes are locked to the stimulus with
Figure 6. Comparison of Actual and Predicted Firing Events for the Salamander
Strong OFF Cell of Figure 5A
(A) Trial-averaged event time; the inter-event
interval ⌬T⬘ ⫽ T⬘j⫹1 ⫺ T⬘j in the predicted event
train is plotted against the corresponding interval ⌬T ⫽ Ti⫹1 ⫺ Ti in the actual event train.
Inset: histogram of the time difference Ti ⫺
T⬘j between an actual event i and a matching
predicted event j.
(B) Trial-averaged spike number, predicted
value N⬘ plotted against actual value N. Panels (B)–(D) are 2-dimensional histograms, with
the gray level indicating the number of counts
in each bin.
(C) Standard deviation across trials of the
event time, predicted value V⬘ plotted against
actual value V.
(D) Standard deviation across trials of the
spike number, predicted value S⬘ plotted
against actual value S.
high timing precision (3.5 ms; see Table 1). The model
matches this behavior very closely and predicts correctly the times and spike numbers of almost all firing
events. On a finer time scale, Figure 5A shows that the
duration of predicted events (the time from first to last
spike of an event) is somewhat longer than that of the
actual events. Note that the event duration was not a
criterion in optimizing the model parameters, but if one
includes a term for duration in the error measure, Equation 10, this kind of discrepancy can largely be elimi-
nated (data not shown). A salamander ON cell is illustrated in Figure 5B. These cells fire much less sparsely
than the strong OFF cells (Berry et al., 1997): their firing
events are more frequent, last longer, and vary more in
their timing. Still, the model adjusts to these spiking
statistics and produces spike trains that match the real
ones both qualitatively and quantitatively (Table 1).
An OFF brisk transient cell from the rabbit retina (Figure 5C) had very different light responses. One finds
clear firing events, whose onset is very precise (3.6 ms;
Figure 7. Systematic Error in the Predictions
of the Model Compared to the Variability of
Neuronal Responses
(A) Error in predicting the mean event time.
Each data point is for one cell, with cell types
identified by different markers. The discrepancy between the mean time of a predicted
event and that of the corresponding actual
event was averaged across all the matched
events from that cell, 兩T ⫺ T⬘兩. This is plotted
against the trial-to-trial jitter V of the actual
event time, also averaged over all events.
Dotted line represents equality.
(B) Error in predicting the mean spike number
of an event. As in (A), but plotting the discrepancy 兩N ⫺ N⬘兩 between model and neuron
against the trial-to-trial number jitter S of the
(C) Error in predicting the jitter of the event
time. As in (A), but plotting the discrepancy
兩V ⫺ V⬘兩 between model and neuron against
the neuron’s trial-to-trial timing jitter V.
(D) Error in predicting the jitter of the spike
number. As in (A), but plotting the discrepancy 兩S ⫺ S⬘兩 between model and neuron
against the neuron’s trial-to-trial number jitter S.
Modeling Precise Spike Trains from Visual Neurons
Table 1. Error Measures for the Model’s Performance, Averaged over All Neurons of a Given Cell Type
Cell Type
Number of Cells
V (ms)
S (spikes)
兩T ⫺ T⬘兩
兩N ⫺ N⬘兩
兩V ⫺ V⬘兩
兩S ⫺ S⬘兩
Salamander retina
strong OFF
Rabbit retina
sustained OFF
delayed OFF
Cat retina
V and S denote the neuron’s variability in event timing and spike number, respectively, averaged over all firing events. These set the scale
for precision in the real visual response, on which the accuracy of the model’s prediction should be evaluated. Listed to the right are the
discrepancies between the predicted and actual event trains in terms of timing 兩T ⫺ T⬘兩, spike number 兩N ⫺ N⬘兩, timing variability 兩V ⫺ V⬘兩, and
number variability 兩S ⫺ S⬘兩. In each case, the absolute value of the discrepancy was averaged over all events. Note these error measures are
those used in optimizing the model (Equation 11).
see Table 1), but they contain large numbers of spikes
(often ⬎10), with low trial-to-trial variability (⬍1 spike;
see Table 1). Again, the model captures the timing, spike
number, and overall shape of these firing events quite
accurately. By contrast, a delayed OFF cell (Figure 5D)
responded very sparsely, with a zero firing rate more
than 91% of the time. Note that its firing events align
with a subset of the events of the OFF brisk transient
cell (Figure 5C). The event timing, spike number, and
the variability of those quantities are again predicted
rather well by the model. However, the expanded time
scale in Figure 5D reveals an additional form of trial-totrial variability in these spike trains: a brief cluster with
very constant spike number jitters back and forth in time
by an amount greater than that of the cluster duration.
The firing mechanism of Figure 4A cannot produce this
type of behavior for the following reason: the time of
the first spike in a cluster can jitter from trial to trial
depending on the value of the noise a(t). However, a(t)
varies slowly and remains essentially constant for the
remainder of the event. Therefore, the model will continue firing spikes until the generator potential reaches
its peak (see the large cluster in Figure 3B). Thus, the
spike cluster should terminate at approximately the
same time in each trial, counter to what happens in
Figure 5D. In principle, one could capture this form of
variability by allowing the latency of the filter to fluctuate,
which would produce the same spike cluster with
slightly different timing on each trial.
A Y-type ON ganglion cell from the cat retina (Figure
5E) produced firing events at a much higher rate than
that of the neurons considered so far and with great
timing precision (0.8 ms; Table 1). The model adapts to
these dynamics, and it is difficult to distinguish the set
of simulated spike trains from the real ones. An X-type
ON cell from the cat LGN (Figure 5F) fired events at an
even greater frequency. However, each event contained
fewer spikes on average, and many events were represented on only a subset of the trials. Again, most aspects
of this cell’s behavior are well matched by the predicted
spike trains.
Evaluation of Performance
A more quantitative evaluation of the simulated spike
trains is given in Figure 6 by comparing the actual and
predicted event trains for an individual cell, and a summary over many neurons is shown in Figure 7. The timing
of firing events is reproduced very well by the model
(Figure 6A): in comparing the inter-event intervals in
actual and simulated spike trains, the discrepancies appear negligible relative to the inter-event intervals themselves. In fact, the root-mean-square timing error incurred by the model is only 3.5 ms (Figure 6A inset).
This is comparable to the amount by which this cell’s
event time jittered between trials, 3.9 ms. A similar level
of accuracy was achieved for many cells (Figure 7A;
Table 1). In almost all cases, the model predicted the
timing of firing events with errors comparable to the
trial-to-trial jitter. Exceptions are the brisk transient cells
in the rabbit retina, where the model’s errors are up
to twice as large as the timing variability of the cell.
Remarkably, the firing times of Y cells in both cat retina
and LGN were matched correctly to within 1.1 ms.
The number of spikes in each event was also predicted very accurately (Figure 6B), with the discrepancy
between data and prediction often less than one spike.
For most cells, the error in predicting the spike number
was comparable to the neuron’s variation across trials
(Figure 7B; Table 1). Again, the predictions for rabbit
brisk transient cells fall short of this mark. Note, however, that these firing events contain large numbers of
spikes (e.g., Figure 5C) so that the errors of the prediction still constitute a small fraction of the absolute spike
The model also served to predict the trial-to-trial jitter
of the event time V and of the spike number S. For the
neuron in Figure 6, both the average time jitter and the
average number jitter are matched correctly. Event by
event, there is a positive correlation between the actual
V and the predicted V⬘ (Figure 6C), but little between
the actual S and predicted S⬘ (Figure 6D). However,
the number jitter S simply does not vary much across
events; what variation there is depends largely on how
close the average number of spikes N is to an integer,
with S smaller for near-integral values of N (Figure 4C).
So to match S more accurately, the model would need
to predict N to within a small fraction of a spike. Nevertheless, across all the cells in this study (Figures 7C
and 7D; Table 1), the model reproduced the stochastic
fluctuations V and S correctly to within ⵑ50%.
measured. In exploring whether an accurate prediction
of spike trains is feasible, we used a visual environment
rich in temporal structure but with no spatial variation.
The conclusions in the following sections are subject to
this caveat. Extensions of the approach to more general
conditions will be discussed.
Figure 8. Comparison of Filter Functions across Cell Types
The waveform of the model’s filter F(t), averaged across all cells of
the same type from salamander (A), rabbit (B), and cat (C).
What is the purpose of constructing a mathematical
model for neuronal responses? On a practical level, the
model allows for a concise description of individual neurons. Each cell’s visual response is characterized by a
small set of numbers, the model parameters (Figure 8;
Table 2). These numbers are communicated easily to
another researcher, who may use them to implement a
realistic neuron of that type. For example, simulated
retinal spike trains can serve as input to computational
models of the visual system. In the distant future, the
simulation might even be performed by a neural prosthesis to replace retinal circuitry and drive optic nerve fibers
directly (Humayun et al., 1999).
On a deeper level, an accurate model for neural responses also serves as a statement of the neural code.
It specifies the statistical relationships between stimuli
and responses. Given a successful “forward” description of firing as a function of the stimulus, one can derive
the corresponding “reverse” description: for any given
spike train, what is the probable time course of the light
intensity, as well as the uncertainty in that estimate?
Such a reverse dictionary would specify what visual
features are encoded by the neuron and transmitted to
its postsynaptic targets.
Strictly speaking, one can be sure of this code only
within the particular visual ensemble in which it was
Quality of the Model’s Predictions
In evaluating the accuracy of the model’s performance,
we placed particular value on the prediction of two aspects of the visual response: the time of occurrence of
each firing event and the number of spikes produced in
that event. This choice is not tied to any particular theory
of neural signaling; most would probably agree that it
is important when a cell fires and also how many spikes
it fires. By these criteria, the formalism performs very
well. In most cases, the error the model makes in predicting the average event time is comparable to the
random variation of that event time across trials (Figure
7; Table 1). Similarly, the error in predicting the average
spike number compares well to the jitter of that spike
number across trials. In other words, the prediction one
gets from this model about the timing and spike number
of a cell’s firing events is as reliable as direct observation
of that same neuron on a previous trial with the same
Is this level of accuracy sufficient? In the real world,
the visual system operates exclusively on single trials,
without the luxury of improving resolution by averaging
many responses to identical stimuli. Nor is there much
opportunity to average across equivalent cells, because
neurons in the early visual system tend to tile the visual
field with little redundancy (Wässle and Boycott, 1991;
Meister and Berry, 1999). Consequently, operation of
the visual system under natural conditions does not
require the properties of these neurons to be specified
more precisely than their trial-to-trial fluctuations. To
understand a neuron’s role in visual behavior, we therefore suggest that a model of the light response can be
deemed successful if its systematic errors are as small
as the neuron’s random errors.
A Simple Neural Code
Given the accuracy of the model, one can understand
how these early visual neurons encode the stimulus,
simply by inspecting how the model does it. The qualitative nature of that code seems rather simple (Figure 9).
The neuron is “interested” in just one aspect of the visual
input: at any point in time, the cell only asks to what
extent the recent stimulus looks like the filter F(t). Thus,
F(t) identifies what feature the cell reports. When the
strength g(t) of that feature rises above threshold, the
cell fires. So the timing of the first spike in a firing event
signals when that feature happened. The number of
spikes in the subsequent burst depends on the peak
excursion of g(t). Therefore, the spike number in the
event specifies how much of the feature was present in
the stimulus. This simple interpretation holds strictly when
the neuron’s firing events are separated sufficiently in
time, by several decay times of the after-potential. At
shorter delays from the preceding event, the accumulated after-potential effectively raises the threshold for
feature detection.
Modeling Precise Spike Trains from Visual Neurons
Table 2. The Best-Fit Parameters of the Model, Averaged over Neurons of the Same Cell Type and Quoted as Mean ⫾ Standard Deviation
The filter F(t) (displayed in more detail in Figure 8); the threshold ␪; the decay time of the after-potential ␶P; the amplitude of the after-potential
B; the noise affecting the generator potential ␴a; and the noise affecting the after-potential ␴b. The time constant of the noise, ␶a, was not
optimized for each individual cell, but is stated here for completeness. Note that ␪, B, and ␴a are all expressed relative to the root-meansquare value of the generator potential g(t), and ␴b is expressed as a fraction of B (see Figure 4 and Experimental Procedures).
Remarkably, this description accounts for the responses of many types of neurons, even though their
spike trains at face value have very different appearance
(Figure 5). Thus, the seeming complexity of response
types may reduce to quantitative variants of a single
common description, each identified by a specific set
of model parameters. Table 2 and Figure 8 present the
average parameters obtained for neurons in different
cell classes. Because the simulation algorithm is relatively simple, one can understand the role that each
parameter plays in creating the characteristic spiking
patterns (Figure 5), and we briefly illustrate this correspondence.
The filter F(t) determines what temporal features in
Figure 9. Summary of the Neural Code Implemented by the Model
The filter F(t) encompasses what stimulus feature the spikes represent; the precise onset of a firing event encodes when that feature
happened; and the number of spikes in the subsequent cluster
reports how strong the feature was.
the flickering stimulus are effective in driving the neuron;
a stimulus waveform that matches the time-reverse of
the filter will produce a large output more likely to cross
threshold. The filters derived for the various cell types
(Figure 8) are all biphasic curves, with a strong primary
peak followed by a second peak of opposite sign. Thus,
all these visual neurons are more sensitive to a change
in the light intensity than to a steady maintained level.
The curves also differ in many respects: the sign of the
principal peak determines whether the response is ONtype or OFF-type; the ratio of the two peaks influences
how transient the response is; and the overall time scale
of the curve sets the time scale of the response. The
filters of rabbit and salamander ganglion cells are comparable in width, whereas those of neurons in cat retina
and LGN are considerably faster. The same progression
of time scales is evident in the raster plots of Figure 5.
Whereas F(t) sets the overall time scale of the fluctuations in the generator potential g(t), only some fraction
of those transients will cross the threshold. Thus, the
threshold ␪ determines the sparseness of firing events
in the neuron’s response. Once the threshold is crossed,
the after-potential immediately reduces the input to the
spike generator by an amount B. If the generator potential continues to rise by more than this, subsequent
spikes can be fired. Thus, the parameter B, along with
the threshold ␪, controls the number of spikes fired in
an event. For example, the cat LGN cells, which tended
to fire in single spikes (Figure 5F), had by far the largest
values of B, whereas the rabbit OFF brisk transient cells
(Figure 5C), with many spikes per event, had the smallest
values (Table 2). The after-potential then decays with
time constant ␶p. Because ␶p is usually long compared
to the duration of a firing event, this decay does not
influence the dynamics of firing during an event. Instead,
it determines the duration of partial refractoriness after
a firing event. For example, salamander OFF cells have
a large ␶p, which contributes to making their firing events
sparser than those of ON cells (Figures 5A and 5B).
The trial-to-trial variability of event timing is determined by ␴a, the noise component of the generator po-
tential, and the shape of the filter F(t), which sets the
overall time scale of the firing process. For example,
salamander OFF cells have a faster filter and a smaller
noise component ␴a than those of ON cells (Table 2)
and correspondingly much more reliable spike timing
(V in Table 1). Cat neurons have a fast filter, and consequently low timing jitter, even though ␴a is comparable
to that of salamander neurons. The variability of the
spike number in an event is determined by both ␴a and
the amount of noise that is added with the after-potential
following each spike, B · ␴b. Across cell types, the ratio
of the two terms varies a great deal, but the second term
often dominates over the first. In practice, therefore, ␴a
determines timing precision and ␴b most of the number
Interpretation of the Model
On a formal level, the spike train simulator of Figure 4
is related to the popular “leaky integrate-and-fire” mechanism (Knight, 1972; Fohlmeister et al., 1977; Lankheet
et al., 1989; Reich et al., 1998) as follows. One can view
h(t) as the membrane potential of a neuron with capacitance C and leak resistance R ⫽ ␶P/C. The capacitance
gets charged by a synaptic current, obtained by passing
the stimulus s(t) through a linear filter. When h(t) crosses
the firing threshold, the cell fires a spike. As a byproduct
of the spike, the membrane is discharged briefly. This
reduces the membrane potential by an amount B, which
subsequently decays with time constant ␶P ⫽ RC. In this
picture, the noise source a(t) represents noise in the
synaptic input, and b(t) is noise in the membrane machinery that affects the degree of discharge. Reich and
colleagues (Reich et al., 1997; Reich et al., 1998) analyzed the firing statistics of such a noisy leaky integrateand-fire model and concluded that it was consistent with
the observed spiking statistics of cat retinal ganglion cell
responses. Here we have shown that by providing the
appropriate input to such a spike generator, one can
in fact predict the neuron’s entire spike train from the
In this picture, the model parameters acquire specific
biophysical meaning. The linear filter comes to represent
all of retinal processing and dendritic integration involved in producing the cell’s synaptic current. The
shape of this function likely reflects the time course of
phototransduction in cones, with additional high-pass
filtering at subsequent retinal synapses (Warland et al.,
1997). The threshold and after-potential components
would represent the neuron’s spike-generating machinery (Kistler et al., 1997). In reality, the separation between
linear and nonlinear processing is probably not quite as
sharp: for example, intracellular recordings from ganglion cells show that the membrane potential is not a
linear function of the stimulus (Baylor and Fettiplace,
1979). During stimulation with random flicker, the generator potential already exhibits discrete depolarizing
transients, which in turn cause the clusters of spikes in
firing events (Sakuranaga et al., 1987). Furthermore, the
model is equally successful with LGN neurons as with
ganglion cells, even though the former receive their input
from the latter. Thus, it is plausible that both the linear
and nonlinear stages of the model are somewhat distributed throughout the circuit and do not map uniquely
onto discrete circuit elements. To learn more about this
correspondence, it will be useful to record signals from
retinal interneurons, such as bipolar and amacrine cells,
and compare them to the internal variables of the model.
Choice of Formalism
There have been many attempts to capture the mathematical relationship between a sensory stimulus and
neuronal firing. With rare exceptions, they seek to predict the neuron’s average firing rate 具r(t)典, a continuous
function of time measured by accumulating the PSTH
over many trials. Generally the firing rate is cast as a
functional of the stimulus 具r(t)典 ⫽ R[s(t);␣i] and then the
parameters ␣i are optimized to approximate the measured relationship between 具r(t)典 and s(t). Several efficient methods have been developed to find a good functional R[ ]. The most systematic approach writes R[ ] as
a Wiener kernel expansion in the stimulus; its parameters are the Wiener kernels of the response, which can be
computed by correlating the response with the stimulus
(Marmarelis and Marmarelis, 1978; Victor and Shapley,
1980; Sakai et al., 1988). Another method—less general,
but often more efficient—writes R[ ] as a sequence of
elementary operations performed on the stimulus. Each
stage in such a “cascade model” performs a simple task
on its input, such as temporal filtering or spatial pooling,
or a nonlinear memory-less transform (Rodieck, 1965;
Victor and Shapley, 1979; Hunter and Korenberg, 1986;
Sakai and Naka, 1987; Victor, 1987; Sakai and Naka,
1995; Benardete and Kaplan, 1997; reviewed in Meister
and Berry, 1999).
The present approach is somewhat different. The
model implements an explicit point process that predicts a spike train with specific spike times. We chose
this form for two reasons:
(1) The average firing rate 具r(t)典 is not really a continuous function. These visual neurons—at least when
driven with rich stimuli such as random flicker or natural
scenes—fire in discrete clusters that are timed reproducibly from trial to trial. Consequently, the average
firing rate 具r(t)典 is exactly zero much of the time but
rises and falls sharply within a few milliseconds at welldefined instants (Figures 1 and 5) (Berry et al., 1997). It
is difficult to model such a series of sharp transients
with a formalism designed for continuous functions. By
contrast, the formalism of Figure 4A produces such firing
events naturally. In the limit of very low noise, spikes
will happen at precisely the same time on every trial;
with very high noise, the spike times will vary a great
deal, and one obtains a smoothly varying firing rate.
(2) It is not sufficient to predict the average firing rate
具r(t)典. To understand what a neuron encodes during sensory processing, one needs to know not only its average
response but also how much the response can vary
about the average (Meister and Berry, 1999). Early visual
neurons can be remarkably precise (Berry et al., 1997;
Reich et al., 1998; Kara et al., 2000), but the amount of
trial-to-trial variability still depends greatly on the cell
type and on the visual stimulus (Figure 7). Thus, a satisfying model of the light response needs to predict not
only the mean firing pattern, but also its stochastic properties. An efficient way of doing that is to model the
stochastic point process itself. By simulating many trials
Modeling Precise Spike Trains from Visual Neurons
and analyzing them in the same way as real neuronal
spike trains, one can predict the average firing rate, but
in addition many other statistics of the response.
Optimizing the Fit
An important component of this modeling strategy is
the method for optimizing the parameters, in particular
the choice of the error measure that gets minimized
during fitting. This expression E (Equations 10–14) evaluates the discrepancy between the actual set of spike
trains and a corresponding predicted set. Its virtue is
that it explicitly contends with the discrete firing events
in the responses. To appreciate this, it helps to consider
a commonly used alternative: in many studies that seek
to predict the time course of a neuron’s firing rate, the
error is measured as the mean squared difference between the actual and the predicted rate (Marmarelis and
Marmarelis, 1978; Victor and Shapley, 1980; Sakuranaga
et al., 1987; Victor, 1987; Korenberg et al., 1988; French
and Korenberg, 1989, 1991; Kondoh et al., 1991; Sakai
and Naka, 1995; Benardete and Kaplan, 1997). Suppose
now that the actual firing rate has a sharp peak, corresponding to a 10 ms wide firing event (e.g., Figure 1B),
and the model with a certain parameter set is able to
predict a similar peak but displaced by 10 ms. Since
the two firing rate functions have no overlap, the mean
squared error is maximal; in fact, the error would be
smaller if the model had not predicted any firing at all.
Even though the 10 ms timing error could be reduced
by a small change in the parameters, it is difficult to
reach that optimum by minimizing the mean squared
error. By contrast, our error measure E would recognize
the correspondence between the actual and predicted
event even if they do not overlap in time. The error then
decreases smoothly with the timing difference, and thus
the model parameters can be optimized successfully.
A crucial step in evaluating this error measure is to
decide how events in the actual response should be
matched to events in the simulation. The algorithm for
finding this correspondence, and indeed the entire form
of E, was heavily inspired by the “distance metric” of
Victor and Purpura (1996). This is a measure to tell how
different two individual spike trains are. It relies on the
simple exercise of turning one spike train into the other
by sliding spikes in time and creating or destroying
spikes. A cost is assigned to each of these elementary
operations, and one can find the transformation with the
lowest overall cost, which in turn is taken as the distance
between the two spike trains. Our distance function E
can be seen as an extension of this approach from single
spikes to firing events. Its flexible form (Equation 10)
allows the model to fit not only the timing of firing events,
but many other properties as well.
Regime of Validity
It will be important to test how far this description of
visual signaling extends. To limit the computational effort, the present study was restricted to temporal variation in the stimulus. The model can certainly be extended
to include spatial processing, for example, by making
the first element a space-time filter F(x,t). This will allow
the prediction of responses to stimuli s(x,t) that vary in
space as well as time. It will also be instructive to explore
very different stimulus environments, including those
that do not produce precise firing events. For example,
when exposed to constant light, retinal ganglion cells
tend to fire at a maintained rate with variable interspike
intervals. If a slow sinusoidal variation is superposed on
the background, the firing rate is modulated smoothly
around the mean (e.g., Enroth-Cugell and Robson,
1966). The model of Figure 4A can certainly replicate
these modes of activity. For example, a negative threshold leads to maintained firing, and a high noise level will
produce variable intervals. However, it is likely that the
same neuron will be described by different sets of parameters under different conditions of stimulation. It will
be instructive to explore how the parameters are modulated by recent visual experience, for example, following
changes in the mean light level or in the average contrast. The compact parametrization of light responses
afforded by the present model promises an equally concise summary of the effects of adaptation. If this can
be achieved, one might obtain an accurate model that
generalizes even to the stimulus ensemble of ultimate
interest: real natural vision.
Experimental Procedures
We presented a random flicker stimulus to the visual system, repeated identically several times, while recording the spike trains of
one or more visual neurons. A mathematical model with 20 parameters was developed to predict spike trains from the stimulus. The
parameters were adjusted to optimize the fit to the real spike trains
over one half of the experiment. Then the quality of the model was
evaluated by how well it predicted responses to different stimuli in
the other half.
Recording and Stimulation
Salamander and Rabbit Retina
We recorded extracellularly from retinal ganglion cells of larval tiger
salamanders and rabbits as described (Meister et al., 1994; Smirnakis et al., 1997). Briefly, the dark-adapted retina was isolated
under dim red light. A piece of retina ⵑ2 mm (salamander) or ⵑ4 mm
(rabbit) in diameter was placed ganglion cell side down on an array
of 61 electrodes in a bath of oxygenated Ringer’s (salamander) or
Ames (rabbit) medium. The electrodes recorded action potentials
from nearby ganglion cells. On occasion, more than one neuron
contributed to the signal on a given electrode; their spike waveforms
were sorted by their shape to obtain reliable single-neuron spike
The stimulus consisted of spatially uniform white light from a
computer monitor. A new intensity was chosen at random every
30 ms from a Gaussian distribution with a standard deviation equal
to 35% of the mean level. The mean light intensity was ⵑ4 mW/m2
at the retina, in the regime of photopic vision. Six different stimulus
segments were used, either 200 s (salamander) or 80 s (rabbit) in
duration. Each of these segments was repeated 12 times, followed
by the next segment. The actual time course of the light emitted by
the monitor was measured using a photocell (Figure 1A), and this
stimulus was used as input for the modeling process.
Cat Retina and LGN
For recordings from cat neurons, the animal was anesthetized (ketamine HCl 20 mg/kg IM, followed by sodium pentothal 20 mg/kg IV
supplemented as needed), ventilated through an endotracheal tube,
and paralyzed with Norcuron (0.3 mg/kg/hr IV). EKG, EEG, temperature, and expired CO2 were monitored continuously. Eyes were refracted, fitted with appropriate contact lenses, and focused on a
tangent screen. For LGN experiments, we performed a 5 mm diameter craniotomy and recorded single neurons in the A laminae with
plastic-coated tungsten electrodes (AM Systems, Everett, WA). For
retinal ganglion cell recordings, the electrode was advanced through
a guide tube that penetrated the sclera. Voltage signals were ampli-
fied, filtered, and digitized (DataWave Discovery software, Longmont, CO). Spikes from single units were sorted by their waveforms
and spike times determined to 0.1 ms resolution.
The stimulus again consisted of spatially uniform white light from
a computer monitor, at a mean luminance of 35 cd/m2 (LGN) or 50
cd/m2 (retina), corresponding to retinal intensities of ⵑ15 mW/m2
and ⵑ21 mW/m2, respectively. A new intensity value was chosen
at random every 7.8 ms, either from a Gaussian distribution with
standard deviation equal to 49% of the mean (LGN) or from a binary
distribution with 100% contrast (retina). Stimulus segments lasted
63.6 s with 6–8 repeats (LGN) or 31.8 s with 4 repeats (retina). The
recorded spike trains did not lock to the video frame rate (128 Hz),
as judged from inspecting their power spectra.
All surgical and experimental procedures were in accordance with
NIH and USDA guidelines and were approved by the Harvard Medical Area Standing Committee on Animals.
Cell Classifications
Salamander retinal ganglion cells were classified into functional
types by the time course of their reverse-correlation to the flicker
stimulus (Warland et al., 1997), in particular the time-to-peak. In
seven preparations, we distinguished the following types: strong
OFF (time-to-peak ⬇ 65 ms), weak OFF (82 ms), and ON (107 ms).
The weak OFF cells generally had firing rates too low for reliable
Rabbit retinal ganglion cells were classified by the criteria of
DeVries (DeVries and Baylor, 1997; DeVries, 1999), using the measured auto-correlation and reverse-correlation functions, and responses to uniform light flashes. Since our conditions of light adaptation were different, these identifications should be regarded as
tentative. In three preparations, we encountered the following types:
ON brisk transient, OFF brisk transient, OFF sustained, OFF delayed,
ON-OFF direction selective.
Neurons in the cat retina and LGN were classified as X- or Y-type
based on their spatiotemporal receptive fields and responses to
contrast-reversing gratings. Putative Y cells had large receptive field
centers and short latencies relative to other neurons at the same
eccentricity. The retinal ganglion cells were also subjected to the
null test of Hochstein and Shapley (1976).
Model to Predict Responses
Here we describe the final form of the model, with attention to
mathematical and computational details. The complete algorithm is
shown schematically in Figure 4A. Formally, the predicted firing rate
of the neuron is given by
r(t ) ⫽ ␦(h(t ) ⫺ ␪) ḣ(t ) H(ḣ(t )),
many as 28 parameters to produce filters with the desired accuracy.
A more efficient choice was made by considering the properties of
the typical filter function (see Figure 8): at short times it has high
amplitude and varies rapidly, but at longer times it tails off gently.
Thus, we used a set of sine functions that were stretched at long
fj(t ) ⫽
冢 冢
冢 冣 冣冣
t 2
, for 0 ⱕ t ⱕ ␶F.
0, otherwise
sin ␲j 2
The maximal length of the filter was ␶F ⫽ 0.95 s (salamander and
rabbit), ␶F ⫽ 0.25 s (cat retina), or ␶F ⫽ 0.19 s (cat LGN). These
functions were then orthonormalized using the Gram-Schmidt
method. In contrast with the Fourier basis, at most 16 of these
components were required for an accurate fit of the filters.
The shape of the after-potential P(t), which lowers h(t) after every
spike and creates a relative refractory period, was chosen as
P(t ) ⫽ Bexp(⫺ t/␶P),
where B and ␶P are free parameters.
The noise signals a(t) and b(t) both have Gaussian amplitude distribution with zero mean and standard deviations ␴a and ␴b, respectively. a(t) has an exponential auto-correlation function with a time
constant of ␶a ⫽ 0.2 s (salamander and rabbit) or ␶a ⫽ 0.02 s (cat).
b(t) is evaluated only at spike times (see Equation 2) and chosen
independently for each spike. ␴a and ␴b are free parameters.
A final parameter is the threshold ␪. Note that increasing ␪ is
equivalent to reducing the amplitudes of F(t), P(t), and a(t) all by the
same factor (see Equations 1 and 2). This redundancy was resolved
by normalizing F(t) such that the filtered stimulus g(t) had a standard
deviation of 1. Thus, the values of ␪, B, and ␴a are all expressed
relative to the root-mean-square amplitude of the stimulus-dependent input to the spike generator g(t). In summary, the 20 parameters
of the model are used as follows: 15 specify the shape of the filter
F(t), one the spike threshold ␪, two the after-potential P(t), and two
the standard deviations of the noise sources a(t) and b(t).
For numerical evaluation, we used discrete time steps of 2 ms
(salamander), 1 ms (rabbit), or 0.2 ms (cat). Simulation on several
trials, using the same stimulus but different noise functions, produced a set of spike trains that was compared with the actual spike
trains on several stimulus repeats. The gradual optimization of the
model parameters was computationally intensive, requiring the
above equations to be evaluated tens of thousands of times. We
used a few notable computational shortcuts. In particular, Equations
3 and 5 were combined to yield
h(t ) ⫽ g(t ) ⫹ a(t ) ⫹
冮⫺∞ r (␶)(1 ⫹ b(␶)) P(t ⫺ ␶)d␶
g(t ) ⫽
g(t ) ⫽
冮⫺∞ s(␶) F(t ⫺ ␶) d␶
H(x) ⫽
冦1,0, ifotherwise.
The function r(t) is a series
兺i ␦(t ⫺ ti) of delta function spikes that
happen at times ti when the generator potential h(t) crosses the
threshold ␪ in the upward direction. h(t) in turn has several components: g(t) is a filtered version of the stimulus time course s(t), obtained by convolution with the filter function F(t). a(t) is a Gaussian
noise source. The final term contains a transient feedback triggered
by each spike, a convolution of the spike process r(t) with the feedback potential P(t), modulated in amplitude by another Gaussian
noise source b(t).
The filter function F(t) defines to a large extent when firing events
will occur, and most of the model’s parameters are dedicated to
adjusting this function. To capture the time course efficiently, we
expanded the filter in an orthonormal basis set,
F(t ) ⫽
兺 kjfj(t ),
j⫽ 1
where the kj are the free parameters. In choosing the basis functions
fj(t), the Fourier basis set was found unsatisfactory, requiring as
兺 kj 冮 s(␶) fj(t ⫺ ␶) d␶,
j⫽ 1
and the integrals were computed only once. Similarly, a set of functions a(t), one for each simulated repetition of the stimulus, were
computed only once for all iterations of the parameter search.
Error Measures
To evaluate the performance of the model, one wants to compare
the set of real spike trains from repeated trials with a set of predicted
spike trains from simulations with different noise waveforms. We
started by parsing each set of spike trains into firing events as
described by Berry and Meister (1998): the average firing rate was
computed from a PSTH across trials (Figure 1B). This histogram
was smoothed with a Gaussian filter whose width was adjusted to
the time scale of variations in the firing rate, as determined from an
auto-correlation of the spike trains (Berry et al., 1997). The resulting
function generally showed sharp peaks separated by valleys of near
zero firing rate. Formally, the boundaries between these firing events
were established by finding minima v of the firing rate which were
significantly lower than the neighboring maxima m1 and m2 such
that 公m1m2/v ⱖ 3 with 95% confidence. The spikes between a pair
of such boundaries were considered part of the same firing event.
For each such firing event, we computed four response statistics
of the distribution across trials:
T ⫽ Average across trials of the time of the first spike,
Modeling Precise Spike Trains from Visual Neurons
matched; the last event in R⬘ is unmatched; or the last events in R
and R⬘ are matched to each other. Letting Ei,j denote the error for
the first i events of R and the first j events of R⬘, the three possibilities
imply that
N ⫽ Average across trials of the spike number in the event,
V ⫽ Standard deviation across trials of the time of the
first spike, and
S ⫽ Standard deviation across trials of the spike
number in the event.
This error measure contains four terms for the discrepancy in each
of the event-specific properties
ET ⫽
兩Ti ⫺ T⬘j 兩
event pairs (i,j)
EN ⫽
兩Ni ⫺ N⬘j 兩 ⫹
event pairs (i,j)
EV ⫽
兺 Ni ⫹ unmatched
兺 N⬘j
events i
兩Vi ⫺ V⬘j 兩
兩Si ⫺ S⬘j 兩,
events j
event pairs (i,j)
EV ⫽
event pairs (i,j)
where the symbols without primes indicate the properties of actual
events, and the symbols with primes those of predicted events. The
final term is proportional to the number of events matched between
the two trains
EM ⫽
In this way a set of spike trains from multiple trials (Figure 1B) was
converted into a single train of events (Figure 1C), each characterized by four numbers.
The next task was to evaluate the discrepancy between the real
and simulated event trains. As illustrated in Figure 1C, some events
in the predicted response clearly match those that occur in the
actual response, but some events in each train may not have corresponding events in the other. To evaluate the quality of this correspondence, one wants to assess whether a given event in one train
has a match in the other train, but also how well the four response
variables (Equation 9) correspond between the actual and the predicted event. Suppose one had decided which events i in the real
train are matched with which events j in the predicted train (by a
method explained below), then we define the overall discrepancy
between the two trains to be
E ⫽ eT ET ⫹ eN EN ⫹ eV EV ⫹ eS ES ⫺ eM EM.
Ei,j ⫽ min(Ei⫺1, j ⫹ eN Ni, Ei, j⫺1 ⫹ eN N⬘j , Ei⫺1, j⫺1 ⫹ Mi,j),
event pairs (i,j)
The constants eX determine the relative importance assigned to
different components of the error. For any given cell, we chose the
particular values eT ⫽ 1/V, eN ⫽ 1/S, eV ⫽ 1/(2V), eS ⫽ 1/(2S), and
eM ⫽ 2, where a bar denotes the average value across all events of
that neuron. The neuron’s averaged timing jitter V is a natural choice
for scaling the errors in predicting the event time T; in effect this
compares the discrepancy between the predicted and actual response to the variation across trials of the actual response. In the
same manner, the averaged spike number jitter S serves to scale
the errors in predicting the spike number N. The coefficients eV and
eS are half as large as eT and eN because we wished to make fitting
of the trial-to-trial variations V and S relatively less important than
fitting the mean properties of an event T and N. Finally, the coefficient eM is a “bonus” for matching two events, and its effects are
discussed further below.
The above measure of error E (Equation 10) assumes that one
knows which events i in the real train should be matched with which
events j in the predicted train. We choose that set of matches which,
in turn, produces the smallest error E. Fortunately, one does not
need to inspect and evaluate all possible correspondences between
the two trains, as long as one observes a natural restriction: that
matches should not cross in time (Figure 1C). Two events in one
train cannot be matched to events in the other train that occur in
the opposite order. With this constraint, one can find the optimal
correspondence between the event trains as follows (Victor and
Purpura, 1996). Consider two event trains R and R⬘. At least one of
the following three possibilities is true: the last event in R is un-
Mi,j ⫽ eT兩Ti ⫺ T⬘j 兩⫹ eN兩Ni ⫺ N⬘j 兩⫹ eV兩Vi ⫺ V⬘j 兩 ⫹
eS兩Si ⫺ S⬘j 兩⫺ eM
is the error incurred in matching the last two events (i,j). The quantities Ei,j can be viewed as a two-dimensional array. Starting with
E0,0 ⫽ 0, this array can be filled recursively using Equation 13.
In practice, the array Ei,j may have more than a thousand elements
on each side, but not all of these need to be computed. For two
events far apart in time—in particular if
兩Ti ⫺ T⬘j 兩 ⬎
(2eNNmax ⫹ eM),
where Nmax is the largest number of spikes in an event in either event
train—the error Mi,j is so large that the possibility of a match need
not be considered. Thus, one needs to evaluate only those Ei,j for
which the times of the last events meet Equation 15 and the array
elements immediately adjacent to those. With this shortcut, the
numerical effort grows proportionally to the total number of events,
rather than its square, and the computation time for this matching
step becomes negligible.
The Initial Guess and Optimization of the Model
A first guess for the various parameters can be obtained by correlating the spike train with the stimulus; this and other analytical properties of the spike generator will be derived in a subsequent paper
(J.K., S. Smirnakis, and M.M., unpublished data). In brief, we computed the spike-triggered average stimulus waveform as well as
the spike-triggered covariance about that average (de Ruyter van
Steveninck and Bialek, 1988). The filter waveform was chosen as
the first eigenvector of this covariance matrix. This analysis also
yielded estimates of the threshold ␪ and the noise standard deviation
␴a. We further chose B ⫽ ␪, ␴b ⫽ ␴a/␪, and ␶P ⫽ 0.2 s (salamander
and rabbit) or 0.02 s (cat).
Starting with this initial guess, the model was used to simulate a
set of spike trains, with as many repeats as in the measured responses. Events were identified in both data sets and matched to
each other as described above, yielding the error measure for this
parameter set. Then we performed a search in the space of 20
parameters, repeating this simulation and evaluation at each step,
to find a set that minimized the error. The search was implemented
by Powell’s method (Press et al., 1992), supplemented by simulated
annealing (10,000 steps, reducing the “temperature” from 0.15 to
0.0005 in 10 geometric steps [Press et al., 1992]), which avoids
getting trapped in local minima of the error surface. Minimization for
a larger number of steps or by different methods did not significantly
decrease the final error value.
The last coefficient of the error measure, eM in Equation 10, was
essential for the fitting process. It effectively encourages the matching of two nearby events in the two trains. Suppose there is a firing
event with Ni spikes in the actual response, but the model predicts
no event nearby, thus the contribution to the error is eNNi. Now we
consider a small change in parameters, which leads to the prediction
of an event close to the correct time, and ask whether the algorithm
will match this to the actual event. Because the parameters have
changed only slightly, the generator potential barely crosses threshold, and the predicted number of spikes is small, N⬘j Ni. If the two
events were to remain unmatched, their contribution to the error
would be eN(Ni ⫹ N⬘j). If the two events were matched, the error
from their spike numbers would be only slightly smaller, eN(Ni ⫺ N⬘j).
However, matching the two events introduces additional error terms
in Equation 10 from comparisons of the event time, T, and the
standard deviations S and V. These penalties will outweigh the benefits of matching the two events. In absence of a compensating
reward, the new parameter set would be rejected, and the search
would never explore promising regions of the parameter space.
Adding eM eliminates this problem and allows a “budding” predicted
event to be matched to an actual event.
in simultaneously recorded retinal, thalamic, and cortical neurons.
Neuron 27, 635–646.
Kistler, W.M., Gerstner, W., and van Hemmen, J.L. (1997). Reduction
of Hodgkin-Huxley equations to a single-variable threshold model.
Neural Comput. 9, 1015–1045.
This work was supported by NIH grants EY10020 to M.M. and
EY10115, EY12196, and NS07009 to R.C.R. and P.R. We thank the
members of our laboratories for comments on the manuscript. J.K.
and M.M. designed and carried out experiments on salamander and
rabbit retina, conceived and implemented the model, and wrote this
article. P.R. and R.C.R. designed and carried out experiments on cat
retina and LGN, and participated in their analysis and interpretation.
Received November 16, 2000; revised March 13, 2001.
Bair, W., and Koch, C. (1996). Temporal precision of spike trains in
extrastriate cortex of the behaving macaque monkey. Neural Comput. 8, 1185–1202.
Baylor, D.A., and Fettiplace, R. (1979). Synaptic drive and impulse
generation in ganglion cells of turtle retina. J. Physiol. 288, 107–127.
Benardete, E.A., and Kaplan, E. (1997). The receptive field of the
primate P retinal ganglion cell, II: Nonlinear dynamics. Vis. Neurosci.
14, 187–205.
Knight, B.W. (1972). Dynamics of encoding in a population of neurons. J. Gen. Physiol. 59, 734–766.
Kondoh, Y., Arima, T., Okuma, J., and Hasegawa, Y. (1991). Filter
characteristics of cercal afferents in the cockroach. J. Comp. Physiol. A 169, 653–662.
Korenberg, M.J., French, A.S., and Voo, S.K. (1988). White-noise
analysis of nonlinear behavior in an insect sensory neuron: kernel
and cascade approaches. Biol. Cybern. 58, 313–320.
Lankheet, M.J., Molenaar, J., and van de Grind, W.A. (1989). The
spike generating mechanism of cat retinal ganglion cells. Vision Res.
29, 505–517.
Marmarelis, P.Z., and Marmarelis, V.Z. (1978). Analysis of Physiological Systems: The White-Noise Approach (New York: Plenum Press).
Meister, M., and Berry, M.J. (1999). The neural code of the retina.
Neuron 22, 435–450.
Meister, M., Pine, J., and Baylor, D.A. (1994). Multi-neuronal signals
from the retina: acquisition and analysis. J. Neurosci. Methods 51,
Berry, M.J., and Meister, M. (1998). Refractoriness and neural precision. J. Neurosci. 18, 2200–2211.
Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.P.
(1992). Numerical Recipes in C: The Art of Scientific Computing,
Second Edition (Cambridge: Cambridge University Press).
Berry, M.J., Warland, D.K., and Meister, M. (1997). The structure
and precision of retinal spike trains. Proc. Natl. Acad. Sci. USA 94,
Reich, D.S., Victor, J.D., Knight, B.W., Ozaki, T., and Kaplan, E.
(1997). Response variability and timing precision of neuronal spike
trains in vivo. J. Neurophysiol. 77, 2836–2841.
Buracas, G.T., Zador, A.M., DeWeese, M.R., and Albright, T.D.
(1998). Efficient discrimination of temporal patterns by motion-sensitive neurons in primate visual cortex. Neuron 20, 959–969.
Reich, D.S., Victor, J.D., and Knight, B.W. (1998). The power ratio
and the interval map: spiking models and extracellular recordings.
J. Neurosci. 18, 10090–10104.
Clifford, C.W., and Ibbotson, M.R. (2000). Response variability and
information transfer in directional neurons of the mammalian horizontal optokinetic system. Vis. Neurosci. 17, 207–215.
Reid, R.C., and Alonso, J.M. (1995). Specificity of monosynaptic
connections from thalamus to visual cortex. Nature 378, 281–284.
Dan, Y., Atick, J.J., and Reid, R.C. (1996). Efficient coding of natural
scenes in the lateral geniculate nucleus: experimental test of a computational theory. J. Neurosci. 16, 3351–3362.
de Ruyter van Steveninck, R.R., and Bialek, W. (1988). Real-time
performance of a movement-sensitive neuron in the blowfly visual
system: coding and information transfer in short spike sequences.
Proc. R. Soc. Lond. B 234, 379–414.
Reinagel, P., and Reid, R.C. (2000). Temporal coding of visual information in the thalamus. J. Neurosci. 20, 5392–5400.
Rieke, F., Warland, D., de Ruyter van Steveninck, R.R., and Bialek,
W. (1997). Spikes: exploring the neural code (Cambridge, MA: MIT
Rodieck, R.W. (1965). Quantitative analysis of cat retinal ganglion
cell response to visual stimuli. Vision Res. 5, 583–601.
DeVries, S.H. (1999). Correlated firing in rabbit retinal ganglion cells.
J. Neurophysiol. 81, 908–920.
Sakai, H.M., and Naka, K. (1987). Signal transmission in the catfish
retina. V. Sensitivity and circuit. J. Neurophysiol. 58, 1329–1350.
DeVries, S.H., and Baylor, D.A. (1997). Mosaic arrangement of ganglion cell receptive fields in rabbit retina. J. Neurophysiol. 78, 2048–
Sakai, H.M., and Naka, K. (1995). Response dynamics and receptivefield organization of catfish ganglion cells. J. Gen. Physiol. 105,
Enroth-Cugell, C., and Robson, J.G. (1966). The contrast sensitivity
of retinal ganglion cells of the cat. J. Physiol. 187, 517–552.
Sakai, H.M., Naka, K., and Korenberg, M.J. (1988). White-noise analysis in visual neuroscience. Vis. Neurosci. 1, 287–296.
Fohlmeister, J.F., Poppele, R.E., and Purple, R.L. (1977). Repetitive
firing: a quantitative study of feedback in model encoders. J. Gen.
Physiol. 69, 815–848.
Sakuranaga, M., and Naka, K. (1985). Signal transmission in the
catfish retina. III. Transmission to type-C cell. J. Neurophysiol. 53,
French, A.S., and Korenberg, M.J. (1989). A nonlinear cascade model
for action potential encoding in an insect sensory neuron. Biophys.
J. 55, 655–661.
Sakuranaga, M., Ando, Y., and Naka, K. (1987). Dynamics of the
ganglion cell response in the catfish and frog retinas. J. Gen. Physiol.
90, 229–259.
French, A.S., and Korenberg, M.J. (1991). Dissection of a nonlinear
cascade model for sensory encoding. Ann. Biomed. Eng. 19,
Shadlen, M.N., and Newsome, W.T. (1994). Noise, neural codes and
cortical organization. Curr. Opin. Neurobiol. 4, 569–579.
Hochstein, S., and Shapley, R.M. (1976). Linear and nonlinear spatial
subunits in Y cat retinal ganglion cells. J. Physiol. 262, 265–284.
Smirnakis, S.M., Berry, M.J., Warland, D.K., Bialek, W., and Meister,
M. (1997). Adaptation of retinal processing to image contrast and
spatial scale. Nature 386, 69–73.
Humayun, M.S., de Juan, E., Jr., Weiland, J.D., Dagnelie, G., Katona,
S., Greenberg, R., and Suzuki, S. (1999). Pattern electrical stimulation of the human retina. Vision Res. 39, 2569–2576.
Usrey, W.M., Reppas, J.B., and Reid, R.C. (1999). Specificity and
strength of retinogeniculate connections. J. Neurophysiol. 82, 3527–
Hunter, I.W., and Korenberg, M.J. (1986). The identification of nonlinear biological systems: Wiener and Hammerstein cascade models.
Biol. Cybern. 55, 135–144.
Victor, J.D. (1987). The dynamics of the cat retinal X cell centre. J.
Physiol. 386, 219–246.
Kara, P., Reinagel, P., and Reid, R.C. (2000). Low response variability
Victor, J.D., and Shapley, R.M. (1979). The nonlinear pathway of Y
ganglion cells in the cat retina. J. Gen. Physiol. 74, 671–689.
Modeling Precise Spike Trains from Visual Neurons
Victor, J., and Shapley, R. (1980). A method of nonlinear analysis in
the frequency domain. Biophys. J. 29, 459–483.
Victor, J.D., and Purpura, K.P. (1996). Nature and precision of temporal coding in visual cortex—a metric-space analysis. J. Neurophysiol. 76, 1310–1326.
Vinje, W.E., and Gallant, J.L. (2000). Sparse coding and decorrelation
in primary visual cortex during natural vision. Science 287, 1273–
Warland, D.K., Reinagel, P., and Meister, M. (1997). Decoding visual
information from a population of retinal ganglion cells. J. Neurophysiol. 78, 2336–2350.
Wässle, H., and Boycott, B.B. (1991). Functional architecture of the
mammalian retina. Physiol. Rev. 71, 447–480.
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Related manuals

Download PDF