Li and DiCarlo (2008) Unsupervised natural experience rapidly alters invariant object representation in visual cortex

Li and DiCarlo (2008) Unsupervised natural experience rapidly alters invariant object representation in visual cortex
REPORTS
1502
proved to be very sensitive to this treatment [median
inhibitory concentration (IC50) < 200 nM], whereas MDA-MB453 cells were relatively resistant
(IC50 > 2 mM) (Fig. 4A). In nude mouse xenografts, groups of five mice were injected with both
cell lines, one on each flank, and were treated by
intraperitoneal injection with rapamycin over an
11-day period. The SUM149PT cells showed a
relative decrease in size followed by stable tumor
growth, whereas the MDA-MB453 cells were
relatively unaffected by treatment (Fig. 4B).
An additional set of 10 breast cancer cell lines
was treated with rapamycin at concentrations of
200 and 400 nM. Cells with deletion or mutation
of FBXW7 (HBL100, 600MPE, SUM149PT,
HCC3153, and HCC1143) or PTEN (HCC1937
and HCC3153) showed significant sensitivity to
killing by rapamycin, although the magnitude of
the effect varied (17) (Fig. 4C). To establish a
direct link between loss of FBXW7 and rapamycin sensitivity, we down-regulated expression levels of FBXW7 using short hairpin RNA (shRNA)
(18) in the rapamycin-resistant MDA-MB453 cells,
which resulted in an increase in sensitivity to this
drug [IC50< 0.8mM (Fig. 4D)].
Our findings implicate FBXW7 in an evolutionarily conserved pathway that controls regulation of mTOR protein levels. Because FBXW7
is a haploinsufficient tumor suppressor that undergoes heterozygous loss in a substantial proportion
of human tumors, the data suggest new approaches
to reduce mTOR levels in cancers by the use of drugs
that may reactivate the remaining copy of FBXW7
in a similar way that nutlins (small-molecule MDM2antagonists) have been shown to activate wild-type
copies of p53 in human tumors (19). Loss of FBXW7
may also be a useful biomarker for sensitivity of
human tumors to inhibitors of the mTOR pathway.
References and Notes
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
M. Welcker, B. E. Clurman, Nat. Rev. Cancer 8, 83 (2008).
S. Akhoondi et al., Cancer Res. 67, 9006 (2007).
J. H. Mao et al., Nature 432, 775 (2004).
H. Rajagopalan et al., Nature 428, 77 (2004).
M. Yada et al., EMBO J. 23, 2116 (2004).
M. Welcker et al., Proc. Natl. Acad. Sci. U.S.A. 101, 9085
(2004).
Z. Kemp et al., Cancer Res. 65, 11361 (2005).
Single-letter abbreviations for the amino acid residues
are as follows: A, Ala; C, Cys; D, Asp; E, Glu; F, Phe; G,
Gly; H, His; I, Ile; K, Lys; L, Leu; M, Met; N, Asn; P, Pro; Q,
Gln; R, Arg; S, Ser; T, Thr; V, Val; W, Trp; Y, Tyr; and X,
any amino acid.
G. Wu et al., Mol. Cell. Biol. 21, 7403 (2001).
H. Strohmaier et al., Nature 413, 316 (2001).
R. M. Neve et al., Cancer Cell 10, 515 (2006).
J. Climent et al., Cancer Res. 67, 818 (2007).
K. Chin et al., Cancer Cell 10, 529 (2006).
J. Fridlyand et al., BMC Cancer 6, 96 (2006).
J. H. Mao et al., Oncogene 22, 8379 (2003).
A. Di Cristofano, B. Pesce, C. Cordon-Cardo,
P. P. Pandolfi, Nat. Genet. 19, 348 (1998).
L. S. Steelman et al., Oncogene 27, 4086 (2008).
M. Welcker, A. Orian, J. E. Grim, R. N. Eisenman,
B. E. Clurman, Curr. Biol. 14, 1852 (2004).
J. K. Buolamwini et al., Curr. Cancer Drug Targets 5, 57 (2005).
We thank B. Vogelstein for providing us with the
HCT116 WT, HCT116 FBXW7−/−, DLD1 wild-type, and
DLD1 FBXW7−/− cell lines; K.I. Nakayama for providing
Fbxw7 knockout mice and vectors (HA-FBXW7 and
HA-FBXW7DF); and O. Tetsu for vector encoding
HA-ubiquitin. These studies were supported by NCI grant
U01 CA84244 and the U.S. Department of Energy
(DE-FG02-03ER63630) to A.B., the University of California
at San Francisco Research-Evaluation Allocation Committee
(REAC) to J.-H.M. A.B. acknowledges support from the
Barbara Bass Bakar Chair of Cancer Genetics.
Supporting Online Material
www.sciencemag.org/cgi/content/full/321/5895/1499/DC1
Materials and Methods
Figs. S1 to S8
Table S1
9 July 2008; accepted 11 August 2008
10.1126/science.1162981
Unsupervised Natural Experience
Rapidly Alters Invariant Object
Representation in Visual Cortex
Nuo Li and James J. DiCarlo*
Object recognition is challenging because each object produces myriad retinal images. Responses
of neurons from the inferior temporal cortex (IT) are selective to different objects, yet tolerant
(“invariant”) to changes in object position, scale, and pose. How does the brain construct this
neuronal tolerance? We report a form of neuronal learning that suggests the underlying solution.
Targeted alteration of the natural temporal contiguity of visual experience caused specific changes
in IT position tolerance. This unsupervised temporal slowness learning (UTL) was substantial,
increased with experience, and was significant in single IT neurons after just 1 hour. Together with
previous theoretical work and human object perception experiments, we speculate that UTL may
reflect the mechanism by which the visual stream builds and maintains tolerant object
representations.
W
hen presented with a visual image,
primates can rapidly (<200 ms) recognize objects despite large variations
in object position, scale, and pose (1, 2). This
12 SEPTEMBER 2008
VOL 321
SCIENCE
ability likely derives from the responses of neurons at high levels of the primate ventral visual
stream (3–5). But how are these powerful
“invariant” neuronal object representations built
www.sciencemag.org
Downloaded from www.sciencemag.org on September 22, 2010
WT cells. Finally, we cotransfected SUM149PT
cells with constructs encoding both FBXW7 and
HA-ubiquitin, and found that ubiquitination of
mTOR was restored by exogenous FBXW7 expression (Fig. 2E). Thus, ubiquitination of mTOR is
largely, if not exclusively, mediated by binding
to FBXW7.
As FBXW7 and PTEN both affect signaling
through mTOR, we examined the genetic status
of both genes in a panel of 53 breast cancer cell
lines (11). Quantitative TaqMan real-time polymerase chain reaction (PCR) assays of the number of copies of FBXW7 and PTEN genes in each
of the cell lines were in good concordance with
data found by bacterial artificial chromosome
(BAC) comparative genomic hybridization (CGH)
microarray (see table S1). Most of the breast cancer cell lines that exhibited loss of a single copy of
FBXW7 (23 out of 53, Fig. 3A) did not show corresponding loss of PTEN. In contrast, of the 14
lines that showed loss of a single copy of PTEN
(Fig. 3A), only one had also lost a copy of FBXW7,
which suggested that FBXW7 and PTEN show
some functional redundancy in tumor development. Similar results were obtained by examination of the copy number status of genomic regions
containing FBXW7 and PTEN genes in three independent human primary breast cancer sets for
which BAC CGH microarray data were available
(12–14). From a total of 450 tumor and cell line
DNA samples shown in Fig. 3, A to D, only 4
had lost a copies of the regions containing both
genes, a result that is unlikely to be a consequence
of random genetic alterations (P = 4.9 × 10−7).
We also considered the possibility that other
somatic changes such as point mutations or genesilencing events could affect the results. The FBXW7
gene continued to be expressed in all 25 breast
cancer cell lines examined (fig. S8), which indicated that no gene silencing had occurred, although very low levels were found in five cell
lines [lanes 10, 13, 14, 16, and 20 (fig. S8)]. All
of these lines had lost one copy of the FBXW7
gene except one (SUM149PT, lane 16), in which
a point mutation was detected (table S1). The
PTEN gene was found to be silent in two cell
lines (fig. S8, lanes 11 and 12), and both had lost
one copy of the PTEN gene. Three mutations in
PTEN were found (fig. S8 and table S1). Thus,
gene silencing (for example, by promoter methylation) or point mutations in FBXW7 and PTEN
are relatively rare mechanisms of inactivation of
these genes, in comparison with single-copy deletions. These data are further compatible with the
identification of both genes as haplo-insufficient
tumor suppressors (3, 15, 16).
Because deletion or mutation of FBXW7 in
human breast cancer cells leads to increased levels
of mTOR, we tested the possibility that cells harboring these deletions may show increased sensitivity to the mTOR inhibitor rapamycin. We treated
two breast cancer cell lines, SUM149PTcells (homozygous FBXW7 mutations) and MDA-MB453
cells (wild-type FBXW7) with rapamycin and
counted numbers of viable cells. SUM149PT cells
by the visual system? On the basis of theoretical
(6–11) and behavioral (12, 13) work, one possibility is that tolerance (“invariance”) is learned
from the temporal contiguity of object features
during natural visual experience, potentially in an
unsupervised manner. Specifically, during natural
visual experience, objects tend to remain present
for seconds or longer, while object motion or
viewer motion (e.g., eye movements) tends to
cause rapid changes in the retinal image cast by
each object over shorter time intervals (hundreds
of ms). The ventral visual stream could construct
a tolerant object representation by taking advantage of this natural tendency for temporally contiguous retinal images to belong to the same
object. If this hypothesis is correct, it might be
possible to uncover a neuronal signature of the
underlying learning by using targeted alteration
of those spatiotemporal statistics (12, 13).
To look for such a signature, we focused on
position tolerance. If two objects consistently
swapped identity across temporally contiguous
changes in retinal position then, after sufficient
experience in this “altered” visual world, the
visual system might incorrectly associate the
neural representations of those objects viewed at
different positions into a single object representation (12, 13). We focused on the top level of the
primate ventral visual stream, the inferior temporal cortex (IT), where many individual neurons
McGovern Institute for Brain Research and Department of
Brain and Cognitive Sciences, Massachusetts Institute of
Technology, Cambridge, MA 02139, USA.
*To whom correspondence should be addressed. E-mail:
dicarlo@mit.edu
possess position tolerance—they respond preferentially to different objects, and that selectivity is
largely maintained across changes in object retinal position, even when images are simply presented to a fixating animal (14, 15).
We tested a strong, “online” form of the temporal contiguity hypothesis—two monkeys visually explored an altered visual world (Fig. 1A,
“Exposure phase”), and we paused every ~15
min to test each IT neuron for any change in
position tolerance produced by that altered experience (Fig. 1A, “Test phase”). We concentrated on each neuron’s responses to two objects
that elicited strong (object “P”, preferred) and
moderate (object “N”, nonpreferred) responses,
and we tested the position tolerance of that object
selectivity by briefly presenting each object at 3°
above, below, or at the center of gaze (16) (fig.
S1). All neuronal data reported in this study were
obtained in these test phases: animal tasks unrelated to the test stimuli; no attentional cueing;
and completely randomized, brief presentations
of test stimuli (16). We alternated between these
two phases (test phase ~5 min; exposure phase
~15 min) until neuronal isolation was lost.
To create the altered visual world (“Exposure
phase” in Fig. 1A), each monkey freely viewed
the video monitor on which isolated objects
appeared intermittently, and its only task was to
freely look at each object. This exposure “task” is
a natural, automatic primate behavior in that it
requires no training. However, by means of realtime eye-tracking (17), the images that played out
on the monkey’s retina during exploration of this
world were under precise experimental control
(16). The objects were placed on the video
Fig. 1. Experimental
design and predictions.
(A) IT responses were
tested in “Test phase”
(green boxes, see text),
which alternated with
“Exposure phase.” Each
exposure phase consisted of 100 normal
exposures (50 P→P, 50
N→N) and 100 swap
exposures (50 P→N, 50
N→P). Stimulus size was
1.5° (16). (B) Each box
shows the exposurephase design for a single neuron. Arrows show
the saccade-induced temporal contiguity of retinal images (arrowheads
point to the retinal images occurring later in
time, i.e., at the end of
the saccade). The swap
position was strictly alternated (neuron-by-neuron) so that it was counterbalanced across neurons. (C) Prediction for responses collected in the test phase:
If the visual system builds tolerance using temporal contiguity (here driven by
saccades), the swap exposure should cause incorrect grouping of two different
www.sciencemag.org
SCIENCE
monitor so as to (initially) cast their image at
one of two possible retinal positions (+3° or −3°).
One of these retinal positions was pre-chosen for
targeted alteration in visual experience (the
“swap” position; counterbalanced across neurons) (Fig. 1B) (16); the other position acted as a
control (the “non-swap” position). The monkey
quickly saccaded to each object (mean: 108 ms
after object appearance), which rapidly brought
the object image to the center of its retina (mean
saccade duration 23 ms). When the object had
appeared at the non-swap position, its identity
remained stable as the monkey saccaded to it,
typical of real-world visual experience (“Normal
exposure”, Fig. 1A) (16). However, when the
object had appeared at the swap position, it was
always replaced by the other object (e.g., P→N)
as the monkey saccaded to it (Fig. 1A, “Swap
exposure”). This experience manipulation took
advantage of the fact that primates are effectively
blind during the brief time it takes to complete a
saccade (18). It consistently made the image of
one object at a peripheral retinal position (swap
position) temporally contiguous with the retinal
image of the other object at the center of the
retina (Fig. 1).
We recorded from 101 IT neurons while the
monkeys were exposed to this altered visual
world (isolation held for at least two test phases;
n = 50 in monkey 1; 51 in monkey 2). For each
neuron, we measured its object selectivity at each
position as the difference in response to the two
objects (P − N; all key effects were also found
with a contrast index of selectivity) (fig. S6). We
found that, at the swap position, IT neurons (on
average) decreased their initial object selectivity
Downloaded from www.sciencemag.org on September 22, 2010
REPORTS
object images (here P and N). Thus, the predicted effect is a decrease in object
selectivity at the swap position that increases with increasing exposure (in the
limit, reversing object preference), and little or no change in object selectivity at
the non-swap position.
VOL 321
12 SEPTEMBER 2008
1503
REPORTS
1504
(Fig. 2C) (“position × exposure” interaction: P =
0.03, one-tailed bootstrap; P = 0.014, one-tailed
permutation test; n = 10). Furthermore, the MUA
object selectivity change at the swap position
continued to increase as the animal received
even more exposure to the altered visual world,
followed a very similar time course in the rate
of object selectivity change (~5 spikes/s per
400 exposures) (Fig. 3C), and even showed a
slight reversal in object selectivity (N > P in
Fig. 4D).
Our main results were similar in magnitude
(Fig. 3, A and B) and statistically significant in
each of the two monkeys (monkey 1: P = 0.019;
monkey 2: P = 0.0192; one-tailed t test). Each
monkey performed a different task during the test
phase (16), suggesting that these neuronal
changes are not task dependent.
Downloaded from www.sciencemag.org on September 22, 2010
for P over N, and this change in object selectivity
grew monotonically stronger with increasing
numbers of swap exposure trials (Fig. 2, A and
C). However, the same IT neurons showed (Fig.
2A) no average change in their object selectivity
at the equally eccentric control position (nonswap position), and little change in their object
selectivity among two other (nonexposed) control objects (see below).
Because each IT neuron was tested for different amounts of exposure time, we first computed a net object selectivity change, ∆(P − N), in
the IT population by using the first and last
available test phase data for each neuron. The
prediction was that ∆(P − N) should be negative
(i.e., in the direction of object preference reversal), and greatest at the swap position (Fig. 1C).
This prediction was borne out (Fig. 3A). The
position specificity of the experience-induced
changes in object selectivity was confirmed by
two different statistical approaches: (i) a direct
comparison of ∆(P − N) between the swap and
non-swap position (n = 101; P = 0.005, onetailed paired t test); (ii) a significant interaction
between position and exposure—that is, object
selectivity decreased at the swap position with
increasing amounts of exposure [P = 0.009 by
one-tailed bootstrap; P = 0.007 by one-tailed
permutation test; tests were done on (P − N)].
The changes in object selectivity at the swap
position were also largely shape-specific. For 88
of the 101 neurons, we monitored the neuron’s
selectivity among two control objects not shown
to the animals during the exposure phase (chosen
similar to the way in which the P and N objects
were selected, fully interleaved testing in each
test phase) (16). Across the IT population, control
object selectivity at the swap position did not
significantly change (Fig. 2A), and the swap
object selectivity changed significantly more than
the control object selectivity (Fig. 3B) (n = 88,
P = 0.009, one-tailed paired t test of swap versus
control objects at the swap position).
These changes in object selectivity were substantial (average change of ~5 spikes/s per 400
exposures at the swap position) (Figs. 2C and
3C) and were readily apparent and highly significant at the population level. In the face of
well-known Poisson spiking variability (19, 20),
these effects were only weakly visible in most
single IT neurons recorded for short durations,
but were much more apparent over the maximal
1-hour exposure time that we could hold neurons
in isolation (Fig. 2C, lower panels). To determine
if the object selectivity changes continued to
grow even larger with longer periods of exposure, we next recorded multi-unit activity (MUA)
in one animal (monkey 2), which allowed us to
record from a number of (nonisolated) neurons
around the electrode tip (which all tend to have
similar selectivity) (21, 22) while the monkey
was exposed to the altered visual world for the
entire experiment (~2 hours) (16). The MUA
data replicated the single-unit results—a change
in object selectivity only at the swap position
Fig. 2. Change in the population object selectivity. (A) Mean population object selectivity at the swap
and (equally eccentric) non-swap position, and for control objects at the swap position. Each row of
plots shows effect among all neurons held for at least the indicated amount of exposure (e.g., top row
shows all neurons held for more than 100 swap exposures—including the neurons from the lower
rows). The object selectivity for each neuron was the difference in its response to object P and N. To
avoid any bias in this estimate, for each neuron we defined the labels “P” (preferred) and “N” by
using a portion of the pre-exposure data (10 repetitions) to determine these labels, and the reminder
to compute the displayed results in all analyses using these labels. Though there was, by chance,
slightly greater initial selectivity at the swap position, this cannot explain the position specificity of
the observed change in selectivity (table S2). (B) Mean population object selectivity of 10 multi-unit
sites. Error bars (A and B) are SEMs. (C) Histograms of the object selectivity change at the swap
position, ∆(P − N) = (P − N)post-exposure − (P − N) pre-exposure. The arrows indicate the means of the
distributions. The mean ∆(P − N) at the non-swap position was −0.01, −0.5, −0.9, and −0.9 spikes/s,
respectively. The variability around that mean (i.e., distribution along the x axis) is commensurate
with repeated measurements in the face of known Poisson spiking variability (fig. S11). (D) Object
selectivity changes at the multi-unit sites. The mean ∆(P − N) at the non-swap position was 1.6
spikes/s.
12 SEPTEMBER 2008
VOL 321
SCIENCE
www.sciencemag.org
Because we selected the objects P and N so
that they both tended to drive the neuron (16), the
population distribution of selectivity for P and N
at each position was very broad [95% range:
(−5.7 to 31.0 spikes/s) pooled across position; n =
101]. However, our main prediction assumes that
the IT neurons were initially object-selective (i.e.,
the response to object P was greater than to object
N). Consistent with this, neurons in our population with no initial object selectivity at the center
of gaze showed little average change in object
selectivity at the swap position with exposure
(fig. S5). To test the learning effect in the most
selective IT neurons, we selected the neurons
with significant object selectivity [n = 52 of
101 neurons; two-way analysis of variance
(2 objects × 3 positions), P < 0.05, significant
main object effect or interaction]. Among this
smaller number of object-selective neurons, the
learning effect remained highly significant and
still specific to the swap position (P = 0.002 by
t test; P = 0.009 by bootstrap; P = 0.004 by
permutation test).
To further characterize the response changes
to individual objects, we closely examined the
selective neurons held for at least 300 exposures
(n = 28 of 52 neurons) and the multi-unit sites (n =
10). For each neuron and site, we used linear
regression to measure any trend in response to
each object as a function of exposure time (Fig.
4A). Changes in response to P and N at the swap
position were apparent in a fraction of single
neurons and sites (Fig. 4A), and statistically
significant object selectivity change was
encountered in 12 of 38 (32%) instances (Fig.
4C) (16). Across our neuronal population, the
change in object selectivity at the swap position
was due to both a decreased response to object P
and an increased response to object N (approx-
imately equal change) (Fig. 4B). These response
changes were highly visible in the single-units
and multi-units held for the longest exposure
times (Fig. 4D).
These changes in the position profile of IT
object selectivity (i.e., position tolerance) cannot
be explained by changes in attention or by adaptation (fig. S10). First, a simple fatigue-adaptation
model cannot explain the position specificity of
the changes because, during the recording of each
neuron, each object was experienced equally
often at the swap and non-swap positions (also
see additional control in table S2). Second, we
measured these object selectivity changes with
briefly presented, fully randomized stimuli while
the monkeys performed tasks unrelated to the
stimuli (16), which argues against an attentional
account. Third, both of these explanations predict
response decrease to all objects at the swap
position, yet we found that the change in object
selectivity at the swap position was due to an
increase in response to object N (+2.3 spikes/s
per 400 swap exposures) as well as a decrease
in response to object P (−3.0 spikes/s per 400
swap exposures) (Fig. 4). Fourth, neither possibility can explain the shape specificity of the
changes.
We term this effect “unsupervised temporal
slowness learning” (UTL), because the selectivity changes depend on the temporal contiguity
of object images on the retina and are consistent
with the hypothesis that the natural stability
(slowness) of object identity instructs the learning without external supervision (6–11). Our
current data as well as previous human object
perception experiments (12) cannot rule out the
possibility that the brain’s saccade-generation
mechanisms or the associated attentional mechanisms (23, 24) may also be needed. Indeed, eye-
Fig. 3. Position specificity, object specificity, and time course. (A) Mean object selectivity change,
∆(P − N), at the swap, non-swap, and central (0°) retinal position. ∆(P − N) was computed as in
Fig. 2C from each neuron’s first and last available test phase (mean ~200 swap exposures). The
insets show the same analysis performed separately for each monkey. (B) Mean object selectivity
change for the (exposed) swap objects and (nonexposed) control objects at the swap position. Error
bars (A and B) are SEMs. The swap object selectivity change at the swap position is statistically
significant (*) in the pooled data as well as in individual animals (P < 0.05, one-tailed t test against
0). (C) Mean object selectivity change as a function of the number of swap exposures for all singleunit (n = 101) and multi-unit sites (n = 10). Each data point shows the average across all the
neurons and sites held for a particular amount of time. Gray line is the best linear fit with a zero
intercept; slope is mean effect size: −5.6 spikes/s per 400 exposures. The slope at the non-swap
position based on the same analysis was 0.6 spikes/s (not shown).
www.sciencemag.org
SCIENCE
VOL 321
movement signals are present in the ventral
stream (25, 26). The relatively fast time scale
and unsupervised nature of UTL may allow rapid
advances in answering these questions, systematically characterizing the spatiotemporal sensory statistics that drive it, and understanding
if and how it extends to other types of image
tolerance (e.g., changes in object scale, pose)
(27, 28).
IT neurons “learn” to give similar responses
to different visual shapes (“paired associates”)
when reward is used to explicitly teach monkeys
to associate those shapes over long time scales [1
to 5 s between images; see, e.g., (29, 30)], but
sometimes without explicit instruction (31, 32).
A top-down explanation of the neuronal selectivity changes in our study is unlikely because
animals performed tasks that were unrelated to
the object images when the selectivity was
probed, and the selectivity changes were present
in the earliest part of the IT responses (~100 ms;
fig S4). But UTL could be an instance of the
same plasticity mechanisms that underlie “paired
associate” learning; here, the “associations” are
between object images at different retinal
positions (which, in the real world, are typically
images of the same object). However, UTL may
be qualitatively different because (i) the learning
is retinal position-specific; (ii) it operates over the
much shorter time scales of natural visual
exploration (~200 ms); and (iii) it is unsupervised
in that, besides the visual world, no external
“teacher” was used to direct the learning (e.g., no
association-contingent reward was used, but we
do not rule out the role of internal “teachers” such
as efferent eye-movement signals). These distinctions are important because we naturally receive
orders-of-magnitude more such experience (e.g.,
~108 unsupervised temporal-contiguity saccadic
“experiences” per year of life).
Our results show that targeted alteration of
natural, unsupervised visual experience changes
the position tolerance of IT neurons as predicted
by the hypothesis that the brain uses a temporal
contiguity learning strategy to build that tolerance
in the first place. Several computational models
show how such strategies can build tolerance
(6–11), and such models can be implemented by
means of Hebbian-like learning rules (8, 33) that
are consistent with spike-timing–dependent plasticity (34). One can imagine IT neurons using
almost temporally coincident activity to learn
which sets of its afferents correspond to features
of the same object at different positions. The time
course and task independence of UTL are
consistent with synaptic plasticity (35, 36), but
our data do not constrain the locus of plasticity,
and changes at multiple levels of the ventral
visual stream are likely (37, 38).
We do not yet know if UTL reflects mechanisms than are necessary for building tolerant
representations. But these same experience manipulations change the position tolerance of human object perception—producing a tendency
to, for example, perceive one object to be the
12 SEPTEMBER 2008
Downloaded from www.sciencemag.org on September 22, 2010
REPORTS
1505
Fig. 4. Responses to objects P and N. (A) Response data to object P and N at the swap position
for three example neurons and one multi-unit site as a function of exposure time. The solid line
is standard linear regression. The slope of each line (∆S) provides a measure of the response
change to object P and N for each neuron. Some neurons showed a response decrease to P,
some showed a response enhancement to N, and others showed both (see examples). (B)
Histograms of the slopes obtained for the object-selective neurons/sites tested for at least 300
exposures. The dark-colored bars indicate neurons with significant change by permutation test
(P < 0.05) (16). (C) Histograms of the slopes from linear regression fits to object selectivity (P −
N) as a function of exposure time; units are the same as in (B). Arrow indicates the mean of the
distribution [the mean ∆S(P − N) at the non-swap position was −1.7 spikes/s, P = 0.38]. The
black bars indicate instances (32%; 12 of 38 neurons and sites) that showed a significant
change in object selectivity by permutation test (P < 0.05). Results were very similar when we
discarded neurons and sites with greater initial selectivity at the swap position (fig. S8). (D)
Data from all the neurons and sites that were tested for the longest exposure time. The plot
shows the mean normalized response to object P and N as a function of exposure time (compare
to Fig. 1C; see fig. S3 for data at the non-swap position and for control objects). Error bars (A
and D) are SEMs.
same identity as another object across a swap
position (12). Moreover, given that the animals
had a lifetime of visual experience to potentially
build their IT position tolerance, the strength of
UTL is substantial (~5 spikes/s change per
hour)—just 1 hour of UTL is comparable to
attentional effect sizes (39) and is more than double that observed in previous IT learning studies
over much longer training intervals (40–42). We
do not yet know how far we can extend this
learning, but just 2 hours of (highly targeted) unsupervised experience begins to reverse the
object preferences of IT neurons (Fig. 4D).
1506
This discovery reemphasizes the importance of
plasticity in vision (4, 32, 35, 37, 40, 41, 43, 44)
by showing that it extends to a bedrock property
of the adult ventral visual stream—positiontolerant object selectivity (45–47), and studies
along the postnatal developmental time line are
now needed.
References and Notes
1. S. Thorpe, D. Fize, C. Marlot, Nature 381, 520 (1996).
2. M. C. Potter, J. Exp. Psychol. (Hum. Learn.) 2, 509
(1976).
3. C. P. Hung, G. Kreiman, T. Poggio, J. J. DiCarlo, Science
310, 863 (2005).
12 SEPTEMBER 2008
VOL 321
SCIENCE
4. N. K. Logothetis, D. L. Sheinberg, Annu. Rev. Neurosci.
19, 577 (1996).
5. R. Q. Quiroga, L. Reddy, G. Kreiman, C. Koch, I. Fried,
Nature 435, 1102 (2005).
6. L. Wiskott, T. J. Sejnowski, Neural Comput. 14, 715
(2002).
7. P. Foldiak, Neural Comput. 3, 194 (1991).
8. G. Wallis, E. T. Rolls, Prog. Neurobiol. 51, 167 (1997).
9. R. Wyss, P. Konig, P. F. Verschure, PLoS Biol. 4, e120
(2006).
10. T. Masquelier, S. J. Thorpe, PLoS Comp. Biol. 3, e31
(2007).
11. T. Masquelier, T. Serre, S. J. Thorpe, T. Poggio, CBCL Tech.
Report #269, Massachusetts Institute of Technology
(2007).
12. D. D. Cox, P. Meier, N. Oertelt, J. J. DiCarlo, Nat.
Neurosci. 8, 1145 (2005).
13. G. Wallis, H. H. Bulthoff, Proc. Natl. Acad. Sci. U.S.A. 98,
4800 (2001).
14. M. Ito, H. Tamura, I. Fujita, K. Tanaka, J. Neurophysiol.
73, 218 (1995).
15. H. Op de Beeck, R. Vogels, J. Comp. Neurol. 426, 505
(2000).
16. Materials and methods are available as supporting
material on Science Online.
17. J. J. DiCarlo, J. H. R. Maunsell, Nat. Neurosci. 3, 814
(2000).
18. J. Ross, M. C. Morrone, M. E. Goldberg, D. C. Burr, Trends
Neurosci. 24, 113 (2001).
19. D. J. Tolhurst, J. A. Movshon, A. F. Dean, Vision Res. 23,
775 (1983).
20. M. N. Shadlen, W. T. Newsome, J. Neurosci. 18, 3870
(1998).
21. K. Tanaka, Cereb. Cortex 13, 90 (2003).
22. G. Kreiman et al., Neuron 49, 433 (2006).
23. T. Moore, M. Fallah, Proc. Natl. Acad. Sci. U.S.A. 98,
1273 (2001).
24. E. Kowler, E. Anderson, B. Dosher, E. Blaser, Vision Res.
35, 1897 (1995).
25. J. L. Ringo, S. Sobotka, M. D. Diltz, C. M. Bunce,
J. Neurophysiol. 71, 1285 (1994).
26. T. Moore, A. S. Tolias, P. H. Schiller, Proc. Natl. Acad. Sci.
U.S.A. 95, 8981 (1998).
27. S. Edelman, S. Duvdevani-Bar, Neural Comput. 9, 701
(1997).
28. G. Wallis, H. Bulthoff, Trends Cogn. Sci. 3, 22 (1999).
29. K. Sakai, Y. Miyashita, Nature 354, 152 (1991).
30. A. Messinger, L. R. Squire, S. M. Zola, T. D. Albright,
Proc. Natl. Acad. Sci. U.S.A. 98, 12239 (2001).
31. Y. Miyashita, Nature 335, 817 (1988).
32. C. A. Erickson, R. Desimone, J. Neurosci. 19, 10404
(1999).
33. W. Gerstner, R. Kempter, J. L. van Hemmen, H. Wagner,
Nature 383, 76 (1996).
34. H. Sprekeler, C. Michaelis, L. Wiskott, PLoS Comp. Biol. 3,
e112 (2007).
35. C. D. Meliza, Y. Dan, Neuron 49, 183 (2006).
36. H. Markram, J. Lubke, M. Frotscher, B. Sakmann, Science
275, 213 (1997).
37. T. Yang, J. H. Maunsell, J. Neurosci. 24, 1617
(2004).
38. Z. Kourtzi, J. J. DiCarlo, Curr. Opin. Neurobiol. 16, 152
(2006).
39. J. H. R. Maunsell, E. P. Cook, Philos. Trans. R. Soc. Lond.
B Biol. Sci. 357, 1063 (2002).
40. C. I. Baker, M. Behrmann, C. R. Olson, Nat. Neurosci. 5,
1210 (2002).
41. E. Kobatake, G. Wang, K. Tanaka, J. Neurophysiol. 80,
324 (1998).
42. N. Sigala, N. K. Logothetis, Nature 415, 318 (2002).
43. E. T. Rolls, G. C. Baylis, M. E. Hasselmo, V. Nalwa,
Exp. Brain Res. 76, 153 (1989).
44. A. Seitz, T. Watanabe, Trends Cogn. Sci. 9, 329
(2005).
45. M. Dill, M. Fahle, Percept. Psychophys. 60, 65
(1998).
46. M. Dill, S. Edelman, Perception 30, 707 (2001).
47. T. A. Nazir, J. K. O’Regan, Spat. Vis. 5, 81 (1990).
48. We thank D. Cox, R. Desimone, N. Kanwisher,
J. Maunsell, and N. Rust for helpful comments
www.sciencemag.org
Downloaded from www.sciencemag.org on September 22, 2010
REPORTS
REPORTS
Supporting Online Material
www.sciencemag.org/cgi/content/full/321/5895/1502/DC1
Materials and Methods
SOM Text
Figs. S1 to 12
Conformational Switch of Syntaxin-1
Controls Synaptic Vesicle Fusion
Stefan H. Gerber,1*† Jong-Cheol Rah,2,3*‡ Sang-Won Min,1*§ Xinran Liu,1,4 Heidi de Wit,5
Irina Dulubova,6 Alexander C. Meyer,3 Josep Rizo,6,7 Marife Arancillo,2 Robert E. Hammer,6,7
Matthijs Verhage,5 Christian Rosenmund,2,3# Thomas C. Südhof1,4,8¶#
During synaptic vesicle fusion, the soluble N-ethylmaleimide-sensitive factor–attachment
protein receptor (SNARE) protein syntaxin-1 exhibits two conformations that both bind to Munc18-1: a
“closed” conformation outside the SNARE complex and an “open” conformation in the SNARE complex.
Although SNARE complexes containing open syntaxin-1 and Munc18-1 are essential for exocytosis, the
function of closed syntaxin-1 is unknown. We generated knockin/knockout mice that expressed only
open syntaxin-1B. Syntaxin-1BOpen mice were viable but succumbed to generalized seizures at 2 to
3 months of age. Binding of Munc18-1 to syntaxin-1 was impaired in syntaxin-1BOpen synapses, and
the size of the readily releasable vesicle pool was decreased; however, the rate of synaptic vesicle fusion
was dramatically enhanced. Thus, the closed conformation of syntaxin-1 gates the initiation of the
synaptic vesicle fusion reaction, which is then mediated by SNARE-complex/Munc18-1 assemblies.
I
ntracellular membrane fusion reactions are
carried out by interactions between SNARE
[soluble N-ethylmaleimide–sensitive factor
(NSF)–attachment protein receptor) and SM
(Sec1-Munc18–like) proteins (1, 2). In Ca2+triggered exocytosis in neurons and neuroendocrine cells, fusion is catalyzed by the formation of
SNARE complexes from syntaxin-1, synaptosomeassociated protein of 25 kDa (SNAP-25), and
synaptobrevin/vesicle-associated membrane protein and the binding of the SM protein Munc181 to these SNARE complexes (1–3). Syntaxin-1
1
Department of Neuroscience, University of Texas Southwestern Medical Center, Dallas, TX 75390–9111, USA.
2
Department of Molecular and Human Genetics and
Department of Neuroscience, Baylor College of Medicine,
Houston, TX 77030, USA. 3Department of Membrane
Biophysics, Max-Planck-Institute for Biophysical Chemistry,
37077 Göttingen, Germany. 4Department of Molecular
Genetics, University of Texas Southwestern Medical Center,
Dallas, TX 75390–9111, USA. 5Department of Functional
Genomics, Vrije Universiteit, 1081 Amsterdam, Netherlands. 6Department of Biochemistry, University of Texas
Southwestern Medical Center, Dallas, TX 75390–9111, USA.
7
Department of Pharmacology, University of Texas Southwestern Medical Center, Dallas, TX 75390–9111, USA.
8
Howard Hughes Medical Institute, University of Texas
Southwestern Medical Center, Dallas, TX 75390–9111, USA.
*These authors contributed equally to this work.
†Present address: Abteilung Innere Medizin III, Universität
Heidelberg, 69120 Heidelberg, Germany.
‡Present address: Developmental Synaptic Plasticity Section, National Institute of Neurological Disorders and
Stroke, Bethesda, MD 20892, USA.
§Present address: University of California, San Francisco,
Mission Bay Campus, San Francisco, CA 94158, USA.
¶Present address: Department of Molecular and Cellular
Physiology and Neuroscience Institute, Stanford University,
Palo Alto, CA 94304–5543, USA.
#To whom correspondence should be addressed. E-mail:
rosenmun@bcm.tmc.edu (C.R.); tcs1@stanford.edu (T.C.S.)
consists of two similar isoforms (syntaxin-1A
and -1B) that are composed of an N-terminal
a-helical domain (the Habc domain) and a Cterminal SNARE motif and transmembrane region. Outside of the SNARE complex, syntaxin-1
assumes a “closed” conformation, in which the
Habc domain folds back onto the C-terminal
SNARE motif (4, 5). In the SNARE complex, by
contrast, syntaxin-1 is “opened” (6). Munc18-1
interacts with syntaxin-1 alone in the closed conformation to form heterodimers (3, 4) and additionally binds to SNARE complexes containing
syntaxin-1 in the open conformation to form
Munc18-1– SNARE complex assemblies (7, 8),
which are essential for exocytosis (3). The
function of the closed conformation of syntaxin1 and its binding to Munc18-1 remain unknown.
We used gene targeting to create mice that
lack syntaxin-1A (syntaxin-1AKO) and contain
the LE mutation in syntaxin-1B, which renders
it predominantly open (syntaxin-1BOpen) (fig. S1)
(9). Studying littermate offspring from crosses
of double-heterozygous syntaxin-1AKO and -1BOpen
mice, we found that homozygous syntaxin-1AKO
mice exhibited no decrease in survival (Fig. 1A)
or other obvious phenotypes (figs. S2 and S3).
The expendability of syntaxin-1A was unexpected in view of its high concentrations and
proposed central functions (10–14) and indicated that syntaxin-1A may be functionally
redundant with syntaxin-1B.
Homozygous mutant syntaxin-1BOpen mice
were also viable but severely ataxic and developed lethal epileptic seizures after 2 weeks of
age (Fig. 1A and fig. S3). The seizure phenotype
of syntaxin-1BOpen mutant mice was recessive
and independent of the syntaxin-1AKO. Thus,
www.sciencemag.org
SCIENCE
VOL 321
Tables S1 and S2
References and Notes
5 May 2008; accepted 15 August 2008
10.1126/science.1160028
syntaxin-1B was selectively essential, probably
because it is more widely expressed than
syntaxin-1A (15). In Caenorhabditis elegans,
transgenic syntaxin-1Open rescues unc-13 mutant
worms from paralysis (16); however, crossing
syntaxin-1BOpen mice with Munc13-1 knockout
mice did not prevent Munc13-1 knockout–
induced death (fig. S4).
The syntaxin-1AKO mutation abolished
syntaxin-1A expression (Fig. 1B), whereas the
syntaxin-1BOpen mutation decreased syntaxin-1B
levels (Fig. 1C). Both mutations produced a
modest decrease in Munc18-1 levels but no major
changes in other proteins (table S1). The syntaxin1Open mutation decreases formation of the
Munc18-1–syntaxin-1 complex but not formation
of SNARE complexes or Munc18-1–SNARE
complex assemblies (fig. S5) (3, 8). Consistent
with this conclusion, less Munc18-1 was coimmunoprecipitated with syntaxin-1 in syntaxin1BOpen mice, whereas other SNARE proteins
coimmunoprecipitated normally (Munc18-1–
SNARE complex assemblies are not stable during
immunoprecipitations, and thus cannot be evaluated) (Fig. 1D and fig. S6).
Electron microscopy of cultured cortical neurons from littermate syntaxin-1BOpen and -1BWT
mice lacking syntaxin-1A revealed increased vesicle docking in syntaxin-1BOpen synapses (~25%
increase) (Fig. 2, A to D). The size of the postsynaptic density also was increased (~20%)
(Fig. 2E), whereas the density of docked vesicles
per active zone length was unchanged (Fig. 2F).
No other structural parameter measured differed
between syntaxin-1BOpen and -1BWT synapses;
in particular, the number and intraterminal distribution of vesicles were unaltered (fig. S7). In
chromaffin cells, however, the syntaxin-1BOpen
mutation caused a large decrease in chromaffin
vesicle docking, similar to that of the Munc18-1
knockout. Again, neither mutation altered the total
number of chromaffin vesicles (Fig. 2, K and L).
Synaptobrevin-2 knockout mice, analyzed in
parallel as a negative control, did not change
chromaffin vesicle docking but did increase the
total number of chromaffin vesicles (Fig. 2L).
Consistent with earlier findings (17–20), these
results indicate that the Munc18-1–syntaxin-1
complex, but not the SNARE complex, functions
in chromaffin vesicle docking. This function
may not be apparent in vertebrate synapses
because active zone proteins that are absent
from chromaffin cells probably dock synaptic
vesicles independent of their attachment to the
Munc18-1–syntaxin-1 complex.
Measurements of spontaneous miniature excitatory postsynaptic currents (mEPSCs), excitatory postsynaptic currents (EPSCs) evoked by
12 SEPTEMBER 2008
Downloaded from www.sciencemag.org on September 22, 2010
and discussion, and J. Deutsch, B. Kennedy,
M. Maloof, and R. Marini for technical support.
This work was supported by the NIH (grant R01EY014970) and The McKnight Endowment Fund for
Neuroscience.
1507
Supporting Online Material
Materials and methods
1. Animals and surgery …………………………………………………………………………………… 2
2. Stimuli presentation and behavior task ………………………………………………………………… 2
3. Neuronal recordings ……………………………………………………………………………………. 3
4. Data analysis ………………………………………………………………………………………….. 4
5. Statistical tests for the "Position x Exposure" interaction ……………………………………………... 6
6. Statistical tests for the response change in single neurons/sites ………………………………………. 7
Supporting text
1. Relationship between the learning effect and IT object selectivity at the center of gaze ……………... 8
2. The object selectivity change is robust to the choice of selectivity metric .…………………………... 8
3. Dependence of effect size on initial object selectivity ………………………………………………… 9
4. The object selectivity change cannot be explained by “adaptation” …………………………………… 10
5. Monte Carlo simulations with Poisson spiking neurons ……………………………………………….. 10
Supporting figures
S1. Monkey tasks during neuronal testing ……………………………………………………………….. 12
S2. Stimuli ………………………………………………………………………………………………... 13
S3. Mean normalized population response to object P and N as a function of exposure time …………... 14
S4. The response time course of the IT selectivity ……………….……………….……………………... 15
S5. Relationship between the learning effect and IT object selectivity at the center of gaze ……………
16
S6. Results in another selectivity metric ..………………………………………………………………..
17
S7. Comparison of the initial object selectivity between the swap and non-swap position ……………...
18
S8. Post-hoc selection to eliminate the initially greater selectivity at the swap position ………………...
19
S9. Population response averages before and after exposure …………………………………………..... 20
S10. Response changes across different time scales for the swap objects ……………………………….
21
S11. Simulations to compare the expectations of Poisson variability and observed variability …………
22
S12. Responses to objects P and N from example neurons in Fig. 4A …………………………………..
23
Supporting tables
S1. Summary of key statistical tests on different subsets of center-of-gaze-selective neurons ………….
24
S2. Post-hoc selection to eliminate the initially greater selectivity at the swap position ………………...
25
Supporting references and notes ………………………………………………………………….. 26
1
Materials and methods
Animals and surgery
Experiments were performed on two male rhesus monkeys (Macaca mulatta, 5.6 and 4.3 kg).
Aseptic surgery was performed to implant a head post and a scleral search coil. After brief
behavioral training (1-3 months), a second surgery was performed to place a recording chamber
(18 mm diameter) to reach the anterior half of the temporal lobe. All animal procedures were
performed in accordance with National Institute of Health guidelines and the Massachusetts
Institute of Technology Committee on Animal Care.
Stimuli presentation and behavioral task
Each recorded neuron was probed in a dimly lit room with a set of 100 achromatic images of
isolated objects (52 natural objects and 48 silhouette white shapes; Fig. S2) presented on a gray
background (21” CRT monitor, 85 Hz refresh rate, ~48cm away, background gray luminance: 22
Cd/m2, max white: 46 Cd/m2). All objects subtended ~1.5º (average of a bounding square and
the filled area). Custom software controlled the stimulus presentation and behavioral monitoring.
Eye position was monitored in nearly real-time using standard sclera coil technique (2), and
saccades >0.2° were reliably detected (1) .
During each Test Phase (~4 minutes), neuronal selectivity at three retinal positions (-3º, 0º, +3º
elevation; all 0º azimuth) was probed in two different tasks. Monkey 1 freely searched an array
of eight small dots (size 0.2º) vertically arranged 3º apart (Fig. S1). The dots never changed in
appearance, but on each "trial", one dot was randomly baited in that a juice reward was given
when the animal foveated that dot, and the next "trial" continued uninterrupted. Typically, the
monkey saccaded from one dot to another (not always the closest dot) looking for the hidden
reward. During this task, objects were presented (100 ms duration) at controlled retinal positions
(onset time was the detected end of a saccade; approximately one such presentation every other
saccade, never back-to-back saccades). The monkey’s task was unrelated to these test stimuli.
To limit unwanted experience across retinal positions, each such presented object was
immediately removed upon detection of any saccade, and these aborted presentations were not
included in the analyses. Monkey 2 performed a more standard fixation task in which it foveated
a single, central dot (size 0.2º, ±1.5º fixation window) while object images were presented at a
natural, rapid rate (5 images/s; 100 ms duration, 100 ms blank intervals). Reward was given at
trial end (5-8 images presented per trial). Upon any break in fixation, any currently present
object image was immediately removed (and not included in the analyses), and the trial aborted.
The animal successfully maintained fixation in 75% of the trials. The presentations before the
broken fixation were included in the analyses. To address possible adaptation concerns, we reperformed the key analysis after discarding the first image presentation in each fixation trial, and
the result was essentially unchanged (also see Fig, S8). Aside from the task differences (freeviewing search vs. fixation), retinal stimulation in the two tasks was essentially identical in that
each tested object image was presented 3º above, below or on the current center of gaze, for 100
ms duration (20-30 pseudo-randomly interleaved repetitions of each). Given equivalent retinal
2
stimulation, IT neuronal responsivity and selectivity are virtually identical when measured
during free viewing and fixation (1). Consistent with this, we found comparable exposureinduced changes in IT object selectivity in each animal (e.g. see Fig. 3).
During each Exposure Phase (~15 minutes), animals freely viewed the monitor while object
images were intermittently (~13 per minute) presented (pseudo-randomly) at +3º and -3º (always
relative to the current center of gaze). The onset of each object was ~65 ms (randomly chosen
between 30-100 ms) after the end of a saccade (defined by eye velocity < 10º/s). Because
foveating a suddenly appearing object is a natural, automatic behavior, essentially no training
was required, and the monkeys almost always saccaded directly to the object (>90%) within 108
ms (median; range: 66-205 ms) and foveated it for >200 ms (after which it was removed and the
monkeys received a drop of juice). For objects presented at the "swap" position (+3º or -3º;
strictly alternated neuron-by-neuron), the object (e.g. "P") was consistently swapped by another
object (e.g. "N") upon the detection of saccade onset (eye speed > 60º/s). We took great care to
ensure the spatiotemporal precision of the stimuli delivery. The to-be-swapped object was
always successfully removed before the end of the saccade, and the new object was present at the
to-be-center of the retina within 1 ms of the saccade end (mean; worst case was 10 ms after
saccade end). To prevent unintended retinal experience, the object image was automatically
taken away if the saccade was not directed toward the object, or if the eye landed more than 1.5º
away from its center.
Neuronal recordings
The extra-cellular spiking activity of single, well-isolated IT neurons was recorded using
standard microelectrode methods (1). 101 neurons were randomly sampled over a ~4x6 mm area
of the ventral STS and ventral surface lateral to the AMTS (Horsey-Clark coordinates: AP 11-15
mm; ML 15-21 mm at recording depth) from the right hemisphere of Monkey 1 and left
hemisphere of Monkey 2. In each daily recording session, we advanced a microelectrode while
the 100 object images (Fig. S2) were pseudo-randomly presented at the center of gaze while the
monkey performed either the free-viewing search task or the fixation task (see Test Phase
above). All responsive neurons with a well-isolated waveform were further probed with the
same object set (initial screening of 100 objects, ~5 repetitions per object, all presented at the
center of gaze).
Main test objects (“swap” objects): Among the objects that drove the neuron "significantly"
above its background response (t-test against 50 ms epoch before stimuli onset, p < 0.05, not
corrected for multiple tests), the most preferred (P) and least preferred (N) objects were chosen
as a pair for the Exposure Phase ("swap objects") subject to the condition that both objects were
either from the "natural" object set or the "silhouette" object set (see Fig. S2). This object
selection procedure aims to find conditions in which both objects drive the neuron, and object P
drives the neuron more strongly than object N. For most neurons, object N was not the second
most preferred object (IT neurons typically respond well to ~10% of object images, which is
consistent with our online observation that N was roughly the tenth most preferred object in the
set of 100 tested objects). Note that, because the procedure for choosing objects P and N was
3
based on limited initial testing, it does not fully guarantee that the selectivity will be found with
further testing (see main text). Furthermore, because the initial testing was at the center-of-gaze
position and IT neurons are not uniformly position tolerant, the procedure also does not
guarantee that the response to P is greater than N at all three tested positions (i.e. possible
"negative" object selectivity at some positions). We use post-hoc analysis and screening to
examine any unexpected effects of this screening procedure (e.g. post-hoc removal of neurons
with low or negative selectivity, table S1). Post-hoc analyses also showed that roughly equal
numbers of neurons were recorded using swap objects from each set (57 natural, 44 silhouette)
and neurons showed virtually the same reported changes in object selectivity when sorted by
object set type.)
Control objects: For each recorded neuron, we also used the initial response testing (above) to
choose a pair of control objects. Our goal was to choose these two objects among which the
neuron was selective, but were very distant from the "swap objects" in shape space. Because
we do not know the dimensions of shape space, we cannot strictly enforce this. In practice, we
simply insured that the control objects were always chosen from the object set that was not used
for the swap objects (i.e. when two objects from the "natural" set were used as "swap" objects,
the two control objects were from the silhouette set, see Fig. S2). Within this constraint, the
control objects were chosen using the exact same responsivity and selectivity criteria as the test
objects (above).
Once the initial screening and object selection were completed, we carried out the Test and
Exposure Phase in alternation for as long as we could maintain isolate of the neuron’s waveform.
Both swap objects and control objects were presented (tested) at all three positions during each
Test Phase but only the swap objects were shown and manipulated during each Exposure Phase.
In addition, multiple-unit activity (MUA) was collected from 10 sites on the IT ventral surface of
Monkey 2 on 10 different experimental sessions (days). Nearby IT neuron have similar object
selectivity (3) and, consistent with this, we have previously shown that MUA is shape selective
and moderately position tolerant (4). MUA was defined as all the signal waveforms in the spike
band (300 Hz – 7 kHz) that crossed a threshold set to ~2 s.d. of the background activity. The
threshold was held constant for the entire session. All other recording procedures were identical
to the recording procedure used for the single-unit recording except: 1) during the Test Phase,
more repetitions (~50) were collected per object image at each position; 2) each Exposure Phase
was approximately twice as long (e.g. 200 "swap" exposures instead of 100).
Data analysis
To get the most statistical power from the data, average firing rates were computed over a time
window optimally chosen for each neuron by an algorithm that estimated neuronal response
onset and offset time (relative to stimulus onset) using all stimuli (see below). Analyses for the
single-unit data in the main text were performed using such neuron-specific spike count windows
(described next). All analyses were also repeated using a standard, fixed spike count window
(onset 100 ms; offset 200 ms) with very similar results (S(P-N) = -5.4 spikes/s per 400
4
exposures at the swap position; -0.3 at the non-swap position; p < 0.01, “position x exposure”
interaction by permutation test).
Neuronal response window: Specifically, for the single-unit data, each neuron’s responses to
each stimulus condition were computed over the same spike count time window. That window
was optimally chosen for each neuron using the following algorithm (5). Given a neuron, we first
computed its average firing rate profiles FRobj(t) in overlapping time bins of 25 ms shifted in
time steps of 1 ms. This averaged firing rate was computed with all the response data to all the
objects presented in all Test Phases for the neuron. We also computed the neuron’s background
rate FRbk as the mean spike rate between 0 and 50 ms after stimulus onset. Because multiple
images were presented on each trial at a relatively rapid rate (see above), the more standard
epoch before each stimulus onset was not ideal for computing background because it was
occupied by the neuron’s response to the previous stimulus. We previously found that the short
epoch starting at stimulus onset and ending before the neuronal response (i.e. before the known
IT latency) provides the most reliable estimate of a “background” firing rate (5). Then, by
subtracting this background rate from the averaged object response rate profile, we obtained an
averaged driven rate profile FRdriven(t) = FRobj(t) - FRbk. Finally, we identified the samples for
which FRdriven(t) was at least 20% of its peak value. The largest continuous interval of samples
fulfilling this requirement was always centered on the peak of the neuronal response. If no other
samples, outside this main “peak” interval, fulfilled the requirement, the extremes of the interval
were chosen as the extremes of the optimal spike count window for that neuron. If the firing
profile exceeded 20% of its peak in other regions of the time axis, these were merged with the
main interval only if they were within 25 ms from it. In this case, the extremes of the merged
interval were chosen as the extremes of the optimal spike count window. In principle this
algorithm could yield a very small response window, so we limited that possibility by imposing a
minimum response window size of 50 ms at the estimated onset latency. (In practice, only two
neurons out of 101 had this minimum window size imposed; neuron1: 30ms; neuron2: 40ms).
All analyses presented in the main text were carried out by counting spikes in these neuronspecific optimized time windows. The mean window start time (± s.d.) was 119.5 ± 38 ms, the
mean window end time was 244 ± 76 ms, and the median duration was 110 ± 50 ms. These time
windows are consistent with previous work (6) and with animal reaction times in recognition
tasks (7).
For the multi-unit data, we used a standard, fixed (100-200 ms) spike count time window for all
recording sites.
For all the results presented in the main text, the object selectivity for an IT neuron was
computed as the difference in its response to object P and N. To avoid any bias in this estimate
of selectivity, for each neuron we defined the labels "P" and "N" by splitting the pre-exposure
response data and used a portion of it (10 response repetitions to each object at each position) to
determine these labels ("P" is the preferred object that elicited a bigger overall response pooled
across position). The label "P" and "N" for the neuron was then held fixed across positions and
later Test Phases. All remaining data were used to compute the selectivity results in the main
text using these labels. This procedure ensured that any observed response difference between
object P and N reflected true selectivity for a neuron, not selection bias.
5
In cases when neuronal response data is normalized and combined (e.g. Figs. 4D, S3, S9), we
used the same normalization scheme through out all the analyses. Specifically, each neuron's
response from each Test Phase was normalized to its mean response to all objects at all positions
in that Test Phase.
Statistical tests for the "Position x Exposure" interaction
A key part of the prediction is that any change in object selectivity should be found
predominantly at the swap position (Fig. 1C). Individual t-tests show a highly significant effect
at the swap position, but no significant effect at the non-swap position (see Fig. 3A). However,
to directly test for an interaction between position and our independent variable (exposure), we
applied a general linear model to the response difference (in firing rate) between the object P and
N. The formulation is similar to the analysis of variance tests (ANOVA). However, it is not
subject to assumptions about the noise distributions.
The model had the following form:
(P N) neuron= n = an + (b1 p) + (b2 e) + (b3 ( p e))
position= p
exp osure= e
The three independent variables of the model are: position (p), exposure (e), and their interaction
(i.e. their product, pe). The position factor has two levels (i.e. p = -1 for swap position, 1 for
non-swap position) the exposure factor has up to 5 levels depending how long a neuron was held,
(i.e. e = 0 for pre-exposure, and can be up to 400 exposures in increments of 100's,). Each an is
the selectivity offset specific to each neuron; b1, b2, and b3 are slope parameters that are shared
among all the neurons. Thus, the complete model for our population of 101 neurons contained a
total of 104 parameters (101 an’s, b1, b2, and b3) that were fitted simultaneously to our entire data
set. The an’s absorb neuron-by-neuron selectivity differences that are not of interest here, and
the remaining three parameters describe the main effects in the population, with b3 of primary
interest (interaction).
To test for a statistically significant interaction, we fit the linear model to the data (standard least
squares), and then asked if the observed value of the interaction parameter (b3) was statistically
different from 0. The validity of the significance was verified by two different approaches: 1)
bootstrap and 2) permutation test.
Bootstrap is widely used to provide confidence intervals on parameter estimates. However, the
bootstrap confidence interval can also be used to provide a significance level for a hypothesis
test (8). To do this, we estimated the distribution of the b3 estimate via a bootstrap over both
neurons and repetitions of each neuron's response data. The exact procedure was done as
follows: for each round of bootstrap over neurons, we randomly selected (with replacement) 101
neurons from our recorded 101 neurons, so a neuron could potentially enter the analysis multiple
times. Once a set of neuron was selected, we then randomly selected (with replacement) the
response repetitions included of each neuron (our unit of data is a scalar spike rate in response to
6
a single repetition of one object at one position). Each neuron’s (P-N) was computed from its
selected response repetitions. The linear model was then fit to the data at the end of these two
random samples to obtain a new b3 value. This procedure was repeated 1000 times yielding a
distribution of b3 values, and the final p-value was computed as the fraction of times that
distribution is less than 0 (p = 0.009, 1000 samples). This p-value is interpreted as: if we were to
repeat this experiment, with both the variability observed in the neuronal responses as well as the
variability in which neurons were sampled, what is the chance that we would not see the
interaction observed here? In effect, the bootstrap analysis determined the confidence interval
around our originally observed b3 value, and the duality of confidence intervals and hypotheses
testing allowed us to report that confidence interval as a p value (8). When the same analysis
was applied to the pair of control objects that was included in the Test Phase but was not shown
during the Exposure Phase, we did not observe a significant interaction (p > 0.05). Simulations
with Poisson spiking neurons have confirmed the correctness of our analysis code.
To further confirm statistical significance level of the interaction (position x exposure) by
permutation test, we created a null distribution by randomly permuting the position labels of the
original data points (swap vs. non-swap) from each exposure level. Following each permutation,
we fit the linear model to the permuted response data. Repeating this, we determined the fraction
of times that the interaction term (b3) was greater the observed value and took this as the p value
(p = 0.007, one-tailed test, 1000 samples). Again, when we applied the same test to the response
data to the control objects, we did not observe a significant interaction (p > 0.05).
Together, our statistical tests point to a position-specific change in object selectivity at the swap
position that increases with amount of exposure. The same statistical tests were applied to the
multi-unit data (n = 10 sites), also yielding a significant interaction between position and
exposure (p = 0.03 bootstrap, p = 0.014 permutation test) for the swap objects. Again no such
interaction was observed for the control objects (p > 0.05 for both bootstrap and permutation
test).
Statistical tests for the response change in single neurons/sites
In Fig. 4B and C, we evaluated each single-unit-neuron/multi-unit-site's response change to P, N
or (P-N) by fitting linear regression as a function of exposure time to obtain a slope (SP, SN,
or S(P-N)). The statistical significance of the response change for each single-unit neuron or
multi-unit site was evaluated by permutation test. Specifically, for each neuron, we randomly
permutated the exposure level label for each response sample, (i.e. which Test Phase each
sample of P and N belongs to). We then re-fit the linear regression to the permuted data and
repeated the same procedure 1000 times yielding a distribution of slopes (“null distribution”).
The p value was determined by counting the faction of time the null distribution exceeded the
linear regression slope obtained from the data. All neuron/sites with p < 0.05 was deemed
significant (dark colored bars in Fig. 4B and C).
7
Supporting text
Relationship between learning effect size and IT neurons’ object selectivity at the center of
gaze
Implicit in our experimental design and hypothesis is that the IT neurons prefer object P over N.
Indeed, object selectivity at the center of gaze may provide the "driving force" for the temporalcontinuity learning in our experiment. Thus, if all our IT neurons had no selectivity among the
objects P and N, then the temporal-continuity learning hypothesis makes no clear prediction in
our data. On average, our recorded IT neurons did have the desired net positive object
selectivity (Table S1). However, even though we aimed to pick objects so that P > N, we used
an initial screen that mainly sought to insure that both objects P and N tended to drive the neuron
(see main text and Materials and methods, above). As a result, the population distribution of
selectivity for P and N at each position was very broad, and some neurons did not have clear
object selectivity at the center of gaze. Here we consider subsets of IT neurons with ever-more
stringent positive object selectivity at the center of gaze by re-performing our key statistical
analyses on these subsets. In sum, the main results are unchanged: 1) as with the entire
population, these subsets of selective neurons show a significant interaction between position
and exposure time, (see details of this statistical test in Materials and methods, above); 2) as with
the entire population, these subsets of selective neurons show a significant shift in their mean
object selectivity (P-N) at the swap position compared to the non-swap position (pair-wise t-test).
These results are summarized in Table S1.
To examine whether a relationship exist between the IT neuron’s selectivity at the center of gaze
and the magnitude of the learning effect, we divided the neurons (n = 101) into sub-populations
based on each neuron’s selectivity for P and N at the center of gaze (Fig S5). Examining the
magnitude of the learning effect across these sub-populations of neurons revealed two things: 1)
neurons with no or negative selectivity, on average, produced no learning effect (effect size = 0.8 spikes/s per 400 exposures); 2) effect size grew increasingly stronger among neurons with
higher initial object selectivity at the center of gaze. This is consistent with the notion that
selectivity at the center of gaze may be a good estimate of the “driving force” for the learning
effect. Expected Poisson spiking variability in the neuronal responses prevented us from
determining if this measure is a perfect predictor of the learning effect size, but we speculate that
the learning effect size in each neuron cannot be simply predicted from object selectivity at the
center of gaze alone, but probably also depends on (e.g.) its initial response magnitudes at the
swap position.
The object selectivity change is robust to the choice of selectivity metric
Is the change in object selectivity reported here robust to the choice of metric quantifying it? In
the main text, we performed all the analyses by quantifying the object selectivity in raw response
difference (spikes/s). Here we explore another standard selectivity metric and show the reported
results is un-affected by the choice of metric.
8
Specifically, for each neuron we computed a “contrast” object preference index (OPI) at each
position using a standard metric (9, 10) :
OPI =
PN
P+N
where P and N is the neuron’s response to object P and N. OPI ranges from -1 to +1 and 0 is no
object selectivity. We computed each neuron’s change in object selectivity magnitude as:
OPI = OPI postexp osure OPI preexp osure
The re-make of main text Fig. 3 using the OPI metric is shown in Fig. S6. To weight all neurons
approximately equally before pooling, the OPI in Fig. S6 was normalized for each neuron to its
pre-exposure object selectivity:
Normalized OPI =
OPI
(OPI preexp osure + 1)
Because the OPI metric ([-1 1]) can have occasionally near-zero or negative values, the addition
of one in the denominator was used to regularize the normalization.
Dependence of effect size on initial object selectivity
We considered the possibility that the observed change in object selectivity at the swap position
(but not at the non-swap position) might somehow have resulted from larger (or smaller) initial
object selectivity magnitude at that position (relative to the non-swap position). Because we
changed the swap and non-swap position across each recorded neuron, position is counterbalanced across neurons so, in the limit, no difference in initial object selectivity magnitude is
expected. Indeed, the initial object selectivity was very closely matched between the swap and
non-swap positions across the neuronal population (Fig S7).
However, even with the counterbalance, it turned out that our recorded population had a slightly
greater initial selectivity at the swap position (by ~ 2 spikes/s in (P-N); Table S2). This
difference between the swap and non-swap position was not significant (p > 0.05, two-tailed ttest). Nevertheless, to fully control for any effect of this slight difference on our results, we
performed a post-hoc analysis to completely match the initial object selectivity. Specifically, we
discarded neurons with significantly greater selectivity (P-N) at the swap position (p < 0.05, onetailed permutation test), and then re-performed the key analyses on the remaining population.
This re-analysis was done for the population of all 101 neurons in Fig. 2 & 3 and for the
population of 38 highly-object-selective neurons/sites in Fig 4C. The results showed that our
post-hoc selection had completely eliminated the small bias in initial selectivity at the two
positions, but the effect size at the swap position in the “equalized” populations was almost as
the original populations (and still no effect at the non-swap position; see Table S2 and Fig. S8).
9
Finally, by sorting the neurons based on their response profile across positions, we found that
even neurons that initially were less responsive at the swap position showed a change in object
selectivity (Fig S9 – especially see panel B).
The object selectivity change cannot be explained by “adaptation”
When repeatedly probed with visual stimuli, neurons in ventral visual stream have been shown to
reduce their evoked responses, a phenomenon referred to as “adaptation” (11-15). Our IT
neuronal data do show evidence of “adaptation” (outlined below). However, the key changes in
IT selectivity we report in this manuscript cannot be explained by “adaptation”. First, although
each object was shown equally often at the two key positions (swap and non-swap positions), the
selectivity change we found was specific to the swap position. This specificity cannot be
explained by any standard model of “adaptation”. Second, approximately half of the selectivity
change was due to an enhanced response to object N (see main text), which is inconsistent with
any fatigue-adaptation model. Third, the selectivity change we found continued to grow larger
and larger for as long as we could measure it (up to two hours),which is suggestive of plasticity,
rather than cortical fatigue.
Consistent with previous reports on IT “adaptation” (11-15), we observed a reduction in the
neurons’ responses over short time scales (i.e. within a trial, ~1 sec) and intermediate time scales
(i.e. within a Test Phase, ~4 min). Such reduction was not specific to objects or position (Fig
S10A and B). However, the change in selectivity we report here emerged slowly over a much
longer times scale (across multiple Test Phases), and was specific to both the swap position and
the object (Fig S10 C). Interestingly, there was virtually no change in response across the longer
time scale over which the learning effect emerged (e.g. non-swap position, Fig. S10 right panel
of row C).
Monte Carlo simulations with Poisson spiking neurons
Beyond the main (net) change in object selectivity reported here, our figures show that many
individual neurons appear to undergo changes in object selectivity in both the predicted and nonpredicted directions (e.g. scatter of the individual neurons in Fig 2C). Is this non-predicted
variability in the observed object selectivity accounted for by Poisson variability (i.e. noise
effects on repeat measurements)?
To address this question, we ran Monte Carlo simulations using Poisson spiking statistics. We
first computed each neuron’s mean firing rates to each object at each position before any
exposure. Using these estimated firing rates, we simulated Poisson spiking neurons. We then
took repeated measurements from this simulated population of neurons. 30-50 response
repetitions were collected from each simulated neuron to constitute a simulated Test Phase,
(matched to the number of response repetitions collected from real neurons/sites). Each
10
simulated neuron was then tested across the same amount of exposure time (i.e. number of Test
Phase) as the real neurons.
In the simulations, we assumed that the object selectivity for P and N at the swap position was
changing at the rate estimated from the data (5.6 spikes/s per 400 exposures, see Fig. 3C) and not
changing at the non-swap position. That is, the simulated neurons’ firing rates at the non-swap
position were held fixed at each simulated Test Phase, while the firing rates at the swap position
were undergoing changes at -0.7 spikes/s for object P and +0.7 spikes/s for object N across each
Test Phase (100 exposures).
This simulation allowed us to determine the expected variability in a real neuronal population’s
selectivity under Poisson firing statistics. Finally, we compared this expected variability to that
observed in our data by plotting the results together (Fig. S11). These Monte Carlo simulations
showed that the magnitude of the non-predicted changes in object selectivity is comparable to
that expected from well-established cortical neuron Poisson spiking statistics (16, 17).
11
Supporting figure S1
Fig. S1. Monkey tasks during neuronal testing. During the Test Phase of the experiment,
Monkey 1 performed a free-viewing search task, searching for a reward "hidden" beneath one of
eight, spatially fixed dots; Monkey 2 performed a standard, fixation task while stimuli were
presented at a rate of 5 per second (100 ms duration, 100 ms gaps). In both cases, retinal
stimulation for the presentation of each object image was identical in that object images were
presented 3° above, 3° below, or at the center of gaze for 100 ms. In both cases, these object
image presentations were fully randomized and unrelated to the animals’ tasks. Each animal also
performed its Test Phase task while we advanced a microelectrode to search for neurons, but
object images were only presented at the center of gaze in that case.
12
Supporting figure S2
Fig. S2. Set of 100 object stimuli. The figure reflects the true relative size of the objects shown
in the experiment. The first 52 images are referred to as the "natural" object set, the latter 48
images as the "silhouette" object set. We only swapped pairs of objects draw from within the
same set (see Materials and methods).
13
Supporting figure S3
Fig. S3. Mean normalized population response to object P and N as a function of exposure time.
(A) Single-unit population data. Each row of plots show neurons held for different amounts of
time, (e.g. the top row shows all neurons held for over 100 swap exposures -- including the
neurons from the lower rows; the second row shows the neurons held for over 200 exposures;
etc). Each neuron's response from each Test Phase was normalized to its mean response to all
objects at all positions in that Test Phase. (B) Multi-unit population data. Note the scale change
on the x-axes between (A) and (B). Though, by chance, there turned out to be slightly greater
initial selectivity at the swap position than the non-swap position, our reported effect was still
strongly present even when post-hoc analysis was used to eliminate this initial selectivity
difference (see Table S2 and Fig S8).
14
Supporting figure S4
Fig. S4. The response time course of the IT selectivity (measured in the Test Phase, see
Materials and methods) before and after at least one hour of experience in the altered visual
world. The plot shows the averaged PSTH difference between object P and N. Colored area
indicates the standard error of the mean. Only the neurons (n = 17) and multi-unit sites (n = 10)
held for the longest exposure time are included in the plot. Each neuron’s PSTH was computed
by smoothing the response data with a Gaussian window (s.d. 10 ms). The responses are aligned
at the onset of the stimulus. The black bar on the top indicates the duration of the stimulus.
Note that the change in selectivity following experience was present even in the earliest part of
the response (~100 ms). The same neurons showed no change for objects presented at the
equally eccentric (non-swap) retinal position (time course not shown; see Figs. 2, 3).
15
Supporting figure S5
Fig. S5. Relationship between the magnitude of the learning effect and the IT neurons’ object
selectivity at the center of gaze. The abscissa shows neurons grouped by the amount of object
selectivity at the center of gaze (“Non-selective neurons” are those with object selectivity (P-N)
less than 1 spike/s; the remaining neurons were split into three even groups). The mean
selectivity among the particular tested two objects (P and N) in the three groups was: 2, 5, and 15
spikes/s. The ordinate shows the mean experience-dependent effect size for the neurons in each
group. Effect size was estimated for each neuron using regression analysis that leverages all the
available data for each neuron (same as in Fig. 4C), so it reflects an unbiased estimate of effect
size that is not confounded by any differences in total duration of exposure. There was no
correlation between this effect size estimate and the duration of exposure (r = 0.05, p = 0.62).
The plot shows the mean effect size in each group of neurons and s.e.m. When the same analysis
was carried out at the non-swap position, the effect sizes were near zero for all four groups of
neurons (not shown).
16
Supporting figure S6
Fig. S6. Results in another selectivity metric. Here, results from all the neurons are re-plotted in
the format of Fig. 3 but in normalized OPI metric. ( ** significantly less than 0 at p < 0.001,
one tailed t-test; N.S. p > 0.05; † approaching significance, p = 0.058).
17
Supporting figure S7
Fig. S7. Initial object selectivity (OPI) of all the neurons at the swap position and non-swap
position. Because the object P and N are determined from the summed response across all
positions using separate data, and IT neurons are not perfectly position-tolerant, the object
selectivity at any given position (swap or non-swap) could have initially negative values. The
inset shows the histogram of the object selectivity difference between the swap and non-swap
positions. Though, by chance, there turned out to be slightly greater initial selectivity at the swap
position than the non-swap position, our reported effect was still strongly present even when
post-hoc analysis was used to eliminate this initial selectivity difference (see Table S2 and Fig
S8). This plot also illustrates the broad distribution of selectivity in the full population, including
some neurons with weak or negative selectivity (see Fig.S5 above).
18
Supporting figure S8
Fig. S8. Post-hoc selection to eliminate the initially greater selectivity at the swap position
among the 38 highly object selective neurons/sites tested for at least 300 exposures (see main
text). (A)(B) Data from the original set of 38 neurons. (A) mean selectivity at the swap and
non-swap position before exposure. (B) is a re-plot of Fig 4C. Red arrow indicates the mean
S(P-N). Black arrow indicates the mean S(P-N) from the non-swap position (individual neural
data not shown). (** significantly less than 0 at p < 0.001, one tailed t-test; n.s. p > 0.05;)
(C)(D) Re-make of (A) and (B) after neurons/sites with significantly greater selectivity at the
swap position were discarded.
19
Supporting figure S9
Fig. S9. Population response averages before and after exposure for those neurons preferring
either the swap or non-swap position. Neurons were sorted by their receptive field profiles.
Neurons in (A) were selected as those for which their maximum response (among either object)
was evoked at the swap position and their minimum response at the non-swap position (n = 14;
max and min taken among all three tested positions). Neurons in (B) were selected as those that
had a maximum response at the non-swap position and minimum response at the swap position
(n = 17). Neurons in (A) and (B) underwent, on average, ~200 exposures (~30 min). Each
neuron's response was normalized to its averaged response to all objects at all positions before
being combined in the group average. Error bars indicate the standard error of the mean.
20
Supporting figure S10
Fig. S10. Response changes across different time scales for the swap objects. (A) Population
mean response changes within a trial (n=111, single-unit data and multi-unit data combined).
For each neuron, the mean responses to each object at each position were subtracted before
averaging. Responses were binned by the order in which the image (object at a position)
appeared in the sequence of stimuli shown in the trial. (B) Population mean response changes
within a Test Phase. For each neuron, the mean responses to each object at each position from
the Test Phases were subtracted before averaging. Responses were binned by the order they
appear in a Test Phase (20 - 30 repeats of each object at each position were tested during each
Test Phase; see Materials and methods). These plots are smoothed with a sliding window (4 data
point wide). (C) Population mean response changes across multiple Test Phases (the left plot in
this row illustrates one view of our reported learning effect). For each neuron, the mean
responses from the first Test Phase (pre-exposure) were subtracted before averaging. Red and
gray traces show responses to object P; blue and black traces show responses to object N.
21
Supporting figure S11
Fig. S11. Simulations to compare the expectations of Poisson spiking variability predictions
with our experimental observed variation in object selectivity. The left panels show the average
effect size (change in object selectivity) at each time point estimated from our data at the swap
position (red bars and arrows, based on 5.6 spikes/s per 400 exposures, see Fig. 3C) and at the
non-swap position (black bars and arrows, assuming no-change, see Fig. 2A). The smooth
curves show the averaged histogram expected under the assumption of Poisson spiking statistics
(using 100 Monte Carlo runs assuming: the same distribution of mean firing rates as that
observed in the recorded population, the same number of response repetitions as that collected in
our experiments: 30 for single-unit, 50 multi-units). The right panel shows the data from the
recorded population. (The red histograms for the single-unit data is a re-plot of Fig. 2C).
Arrows indicate the mean object selectivity changes observed in the data. The over-laid dash
lines are the distributions generated from the simulations (left), simply re-plotted on top of the
data. Note that the dashed curves are quite broad (the effect of Poisson spiking “noise”) and are
approximately matched to the empirical distributions (solid histograms).
22
Supporting figure S12
Fig. S12. Responses to objects P and N from example neurons in Fig. 4A. (A) Response data to
object P and N at the swap position (This is a re-plot of the neurons in Fig. 4A). (B) Response
data from the non-swap position for these neurons. The object P and N are determined from the
summed response across all positions using separate data before exposure. In both panels, the
solid lines are the best-fit linear regression.
23
Supporting table S1
Screen criteria based on
responses at the center of
gaze
Full population (no screen)
n=101
Mean response rates under each
screen (spikes/s)
Center of
Swap
Non-swap
gaze
position
position
P
N
P
N
P
N
20
14
21
16
20
16
Permutation Test for
“position x exposure”
interaction
t-Test swap vs.
non-swap position
*p = 0.007
*p = 0.005
Statistically selective
(i.e. P>N at p<0.05)
n=56
25
16
27
19
25
19
*p = 0.010
*p = 0.023
Statistically selective AND
(P-N)>3 spikes/s
n=48
29
18
29
20
27
20
*p = 0.016
*p = 0.041
Statistically selective AND
(P-N)>5 spikes/s
n=38
33
20
32
21
30
22
*p = 0.039
*p = 0.025
Table S1. Summary of key statistical tests on different subsets of center-of-gaze-selective
neurons. Left table columns show the mean responses of all the neurons in each subset to objects
P and N. The selectivity screens were done after the “P” and “N” labels were determined using
separate data to avoid bias (that is, a positive response difference implies real selectivity for P
over N, not selection bias). The P-N differences are greater at the swap position for all groups of
neurons because a few most foveal selective neurons (3/38) had much greater selectivity at the
swap position. The right columns show the outcome of the two key statistical analyses. These
results are robust to removal of outlier points (Fig 2C top panel, left-most bar). Though the
overall mean effect size per neuron is larger for more selective neurons (i.e. going from the top
to the bottom of the table, see Fig. S5), the statistical significance level (p values) is slightly
smaller (slightly larger p value) due to the drop in the number of neurons.
24
Supporting table S2
Difference in
(P-N) pre-exposure
Effect size
(spikes/s)
Non-Swap position
Swap – Non-swap
(spikes/s)
Swap position
Full population in Fig. 2, 3
(n=101)
2.05
-5.6 *
0.9 (n.s.)
Matched (P-N) pre-exposure
(n=84)
-0.01
-4.2 *
0.73 (n.s.)
Difference in
(P-N) pre-exposure
Effect size
(spikes/s)
Non-Swap position
Swap – Non-swap
(spikes/s)
Swap position
Object-selective
neurons/sites in Fig. 4C
(n=38)
2.70
-8.3 **
-1.7 (n.s.)
Matched (P-N) pre-exposure
(n=33)
-0.34
-6.5 **
-0.1 (n.s.)
Table S2. Summary of the key results after the initially greater selectivity at the swap position
were adjusted for post-hoc. Effect size, S(P–N), was estimated for each neuron using
regression analysis that included all the available data for each neuron (same as in Fig. 4C).
S(P–N) was significantly different from 0 only at the swap position (* p<0.05; ** p<0.001; one
tailed t-Test).
25
Supporting references and notes
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
J. J. DiCarlo, J. H. R. Maunsell, Nat. Neurosci. 3, 814 (2000).
D. A. Robinson, IEEE Transactions on Biomedical Engineering 101, 131 (1963).
K. Tanaka, Cereb Cortex 13, 90 (2003).
G. Kreiman et al., Neuron 49, 433 (2006).
D. Zoccolan, M. Kouh, T. Poggio, J. J. DiCarlo, J. Neurosci. 27, 12292 (2007).
G. C. Baylis, E. T. Rolls, C. M. Leonard, J. Neurosci. 7, 330 (1987).
J. J. DiCarlo, J. H. R. Maunsell, J. Neurophysiol. 89, 3264 (2003).
B. Efron, R. J. Tibshirani, An Introduction to the Bootstrap. (Chapman & Hall, 2003)
C. I. Baker, M. Behrmann, C. R. Olson, Nat. Neurosci. 5, 1210 (2002).
N. Sigala, N. K. Logothetis, Nature 415, 318 (2002).
H. Sawamura, G. A. Orban, R. Vogels, Neuron 49, 307 (2006).
E. K. Miller, L. Li, R. Desimone, Science 254, 1377 (1991).
S. Sobotka, J. L. Ringo, Exp. Brain. Res. 96, 28 (1993).
C. G. Gross, P. H. Schiller, C. Wells, G. L. Gerstein, J. Neurophysiol. 30, 833 (1967).
G. C. Baylis, E. T. Rolls, Exp. Brain. Res. 65, 614 (1987).
D. J. Tolhurst, J. A. Movshon, A. F. Dean, Vision Res. 23, 775 (1983).
M. N. Shadlen, W. T. Newsome, J. Neurosc. 18, 3870 (1998).
26
Unsupervised Natural Experience Rapidly Alters
Invariant Object Representation in Visual Cortex
Nuo Li, et al.
Science 321, 1502 (2008);
DOI: 10.1126/science.1160028
This copy is for your personal, non-commercial use only.
If you wish to distribute this article to others, you can order high-quality copies for your
colleagues, clients, or customers by clicking here.
Permission to republish or repurpose articles or portions of articles can be obtained by
following the guidelines here.
Updated information and services, including high-resolution figures, can be found in the online
version of this article at:
http://www.sciencemag.org/cgi/content/full/321/5895/1502
Supporting Online Material can be found at:
http://www.sciencemag.org/cgi/content/full/321/5895/1502/DC1
This article cites 45 articles, 14 of which can be accessed for free:
http://www.sciencemag.org/cgi/content/full/321/5895/1502#otherarticles
This article has been cited by 11 article(s) on the ISI Web of Science.
This article has been cited by 4 articles hosted by HighWire Press; see:
http://www.sciencemag.org/cgi/content/full/321/5895/1502#otherarticles
This article appears in the following subject collections:
Neuroscience
http://www.sciencemag.org/cgi/collection/neuroscience
Science (print ISSN 0036-8075; online ISSN 1095-9203) is published weekly, except the last week in December, by the
American Association for the Advancement of Science, 1200 New York Avenue NW, Washington, DC 20005. Copyright
2008 by the American Association for the Advancement of Science; all rights reserved. The title Science is a
registered trademark of AAAS.
Downloaded from www.sciencemag.org on September 22, 2010
The following resources related to this article are available online at www.sciencemag.org
(this information is current as of September 22, 2010 ):
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertising