g June13 2013
Neural Networks 17 (2004) 471–510
How laminar frontal cortex and basal ganglia circuits interact
to control planned and reactive saccades
Joshua W. Browna, Daniel Bullockb,c,*, Stephen Grossberg
Department of Psychology, Washington University, Campus Box 1125, St. Louis, MO 63130-4899, USA
Department of Cognitive and Neural Systems, Boston University, 677 Beacon Street, Boston, MA 02215, USA
Center for Adaptive Systems, Boston University, 677 Beacon Street, Boston, MA 02215, USA
Received 30 May 2003; accepted 14 August 2003
How does the brain learn to balance between reactive and planned behaviors? The basal ganglia (BG) and frontal cortex together allow
animals to learn planned behaviors that acquire rewards when prepotent reactive behaviors are insufficient. This paper proposes a new model,
called TELOS, to explain how laminar circuitry of the frontal cortex, exemplified by the frontal eye fields, interacts with the BG, thalamus,
superior colliculus, and inferotemporal and parietal cortices to learn and perform reactive and planned eye movements. The model is
formulated as fourteen computational hypotheses. These specify how strategy priming and action planning (in cortical layers III, Va and VI)
are dissociated from movement execution (in layer Vb), how the BG help to choose among and gate competing plans, and how a visual
stimulus may serve either as a movement target or as a discriminative cue to move elsewhere. The direct, indirect and hyperdirect pathways
through the BG are shown to enable complex gating functions, including deferred execution of selected plans, and switching among
alternative sensory-motor mappings. Notably, the model can learn and gate the use of a What-to-Where transformation that enables spatially
invariant object representations to selectively excite spatially coded movement plans. Model simulations show how dopaminergic reward
and non-reward signals guide monkeys to learn and perform saccadic eye movements in the fixation, single saccade, overlap, gap, and delay
(memory-guided) saccade tasks. Model cell activation dynamics quantitatively simulate seventeen established types of dynamics exhibited
by corresponding real cells during performance of these tasks.
q 2003 Elsevier Ltd. All rights reserved.
Keywords: Basal ganglia; Frontal cortex; Cortical layer; Saccade; Gating; Dopamine; Reinforcement learning; Action selection; Planning; Parkinson’s disease
1. Introduction
This article proposes detailed mechanistic solutions to
several key problems in sensory-motor control: How does
the brain learn to balance between reactive and planned
movements? How do recognition and action representations
in the brain work together to launch movements toward
valued goal objects? How does the brain learn and recall the
myriad movement plans it needs to switch among different
tasks, when each plan may be sensitive to different
combinations of scenic cues and timing constraints?
The article treats these problems by modeling the
saccadic, or ballistic, eye movement system. Solving these
* Corresponding authors. Tel.: þ1-617-353-7858/7857; fax: þ 1-617353-7755.
E-mail addresses: [email protected] (J.W. Brown); [email protected]
cns.bu.edu (D. Bullock); [email protected] (S. Grossberg).
0893-6080/$ - see front matter q 2003 Elsevier Ltd. All rights reserved.
problems requires interactions among multiple brain regions,
including inferotemporal, parietal, and prefrontal cortex
(PFC); basal ganglia (BG); amygdala; cerebellum; and
superior colliculus (SC). In amphibians and all land
vertebrates, the BG interact with a laminar structure, the
optic tectum (OT), or its homolog, the SC, to control
orienting actions and, in some species, prey-catching actions
(Butler & Hodos, 1996; Marin, Smeets, & Gonzalez, 1998).
The mammalian BG also interact with distinct areas of frontal
cortex—also laminar structures—to control orienting, cognitive, and manipulative behaviors (Hikosaka & Wurtz,
1989; Passingham, 1993; Strick, Dum, & Picard, 1995).
Lesions of the BG uniquely cause devastating disorders of the
voluntary movement system, e.g. Parkinsonian akinesia,
Huntington’s chorea, and ballism (Albin, Young, & Penney,
1989). Such observations suggest a tight link between
volitional movement and BG interactions with laminar
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
action control structures, which provide a natural basis for
differentiating between plan activation and plan execution.
Whereas laminar organization has been neglected in most
models of BG function, the present model explains how BG
interactions with laminar target structures satisfy staging and
learning requirements of voluntary behavior.
The model simulates learning and performance of the
saccadic tasks that are summarized in Fig. 1 (cf. Hikosaka,
Sakamoto, & Usui, 1989, p. 781). It is also used to simulate
performance in two related tasks. Recording during such
tasks has produced a wealth of electrophysiological data
that, in concert with anatomical studies, serve as hard
constraints on model development. As a set, these tasks
challenge an animal’s ability to plan, withhold, and generate
goal-directed movements in a way that satisfies instrumental
reward contingencies. Simulations (summarized in
Figs. 9 –11) show that the model shown in Fig. 2 can
learn and perform all the tasks, and can regenerate
seventeen qualitatively distinct types of task-related activation dynamics exhibited by cells in the SC, BG, thalamus,
and oculomotor areas of frontal and parietal cortex.
A key adaptive challenge is to balance reactive and
planned movement (Grossberg & Kuperstein, 1986; Hallett,
Fig. 1. Oculomotor tasks of Hikosaka et al. (1989). Black bars indicate
intervals of visual stimulus presentations and the trace labeled E gives the
horizontal component of eye position (line of gaze). In the fixation task, the
subject must maintain gaze on the fixation point, F, despite a brief display
of a distracter target, T, at a different locus. In the saccade task, the subject
must make a pro-saccade from the fixation point to the target, which
appears at a different locus, just as the fixation point shuts off. In the overlap
task (similar to a GO/NOGO task), the target and the fixation point
are displayed in overlapping intervals. A pro-saccade to the target is
rewarded only if generated after the fixation point shuts off. The gap task
imposes a delay between the offset of the fixation point and the onset of the
target. The gap task target appears at a consistent location across trials, and
the subject learns to make an anticipatory pro-saccade to the target location
during the gap between fixation light offset and target onset. The delay task
requires the subject to remember the location of a briefly-flashed target and
later foveate it. [Adapted with permission from Hikosaka et al., 1989, p.
781.] The model in Fig. 2 learned and performed all these tasks.
1978). Rapid reactive movements are needed to ensure
survival in response to unexpected dangers. Planned
movements often take longer to elaborate. How does the
brain prevent reactive movements from being triggered in
situations where a more slowly occurring planned movement
would be more adaptive? Movement gates can prevent
the reactive movement from being launched until the
planned movement can effectively compete with it. Then a
winning movement command can open its gate and launch its
movement. The proposed model shows how a cooperative set
of physiological and circuit properties: prevent a reactive
movement command from opening the gate before a planned
movement command is ready to open it; allow the
reactive and planned commands to compete for dominance;
yet also allow a reactive movement command to open the
gate when no planned movement command is being formed.
Conditional movements towards valued goal objects
cannot be made until the goal objects are recognized and
movement directions specified. Formidable memory storage
problems would ensue if the brain had to learn separate object
recognition codes for every retinotopic position and size of
an object. To achieve efficient object recognition, the What
cortical processing stream builds object representations that
are ‘positionally invariant’, i.e. independent of the retinotopic position or size of the object (Bar et al., 2001; Sigala &
Logothetis, 2002; Tanaka, Saito, Fukada, & Moriya, 1991).
Given that recognition codes are independent of position,
how does the brain compute how to move to the position of an
object after it is recognized? After eliminating the link
between an object’s identity and position for purposes of
object recognition, the brain needs to re-establish this link for
purposes of movement. The Where cortical processing
stream elaborates the object positions and directions needed
to compute motor commands. The model proposes how
interactions across the What and Where processing streams
overcome their complementary informational deficiencies to
generate movements towards recognized objects.
It is not enough to recognize and move towards an object.
An animal needs to know when to move towards or away
from an object and when not to do so, depending on reward
contingencies. Decision criteria include such stimulus
properties as color, size, shape, motion and the state of the
body, taken individually or in combination. In addition,
when confronted with the same scene, an animal may act
with respect to different objects depending on its changing
needs, such as food if hungry or water if thirsty. The model
explains how the brain learns and remembers many plans
that involve different sets of discriminative and scheduling
constraints, and how it switches among them as needed.
According to the proposed model (Fig. 2), these functional
problems find a mechanistic solution in BG interactions with
the SC and frontal cortex. Reward-related dopaminergic
signals modulate learning in the BGs striatum and the frontal
cortex (Gaspar, Bloch, & Le Moine, 1995; Schultz, 1998).
The trained BG system allows or prevents movements,
according to their appropriateness (Bullock & Grossberg,
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Fig. 2. A laminar model of BG interactions with the FEF and the SC. Separate gray-shaded blocks highlight the major anatomical regions whose roles in
planned and reactive saccade generation are treated in the model. Major anatomical abbreviations, e.g. PPC, are defined in Table 1. Excitatory links are shown
as arrowheads, inhibitory as ballheads, in this and all subsequent figures. Filled semi-circles terminate cortico-striatal and cortico-cortical pathways modeled as
subject to learning, which is modulated by reinforcement-related dopaminergic signals (dashed arrows). In the FEF block, Roman numerals I– VI label cortical
layers; Va and Vb, respectively, are superficial and deep layer V. Further symbols are variable names in the mathematical model. Subscripts xy index
retinotopic coordinates, whereas subscript i denotes an FEF zone gated by an associated BG channel (details in Fig. 5). All variables for FEF activities use the
symbol F. Processed visual inputs Ixy
and Ixyj
emerging from visual areas including V4 and posterior IT feed into the model FEF input cells and affect
activations Fxyi
: Fibers carrying such inputs are predicted to synapse on cells in layer III (and possibly layers II and IV). Visual input also excites the PPC, Pxy ;
and anterior IT, Tj : A PFC motivational signal I ðMÞ arouses PFC working memory activity Ci ; which in turn provides a top-down arousal signal to model FEF
excite FEF planning cells Fxyi
; which are predicted to reside in layers III/Va (and possibly
layer VI cells, with activities FiðGÞ : The FEF input cell activities Fxyi
layer II). Distinct plan layer activities represent alternative potential motor responses to input signals, e.g. a saccade to an eccentric target or to a central fixation
point. FEF layer VI activities FiðGÞ excite the groups/categories of plans associated with GCZs i and associated thalamic zones k. The BG decide which plan to
execute and send a disinhibitory gating signal that allows thalamic activation Vk ; which excites FEF layer Vb output cell activities Fxyi
to execute the plan. The
model distinguishes (Kemel et al., 1988) a thalamus-controlling BG pathway, whose variables are symbolized by B, and a colliculus-controlling pathway,
whose variables are symbolized by G. Thus, the striatal direct (SD) pathway activities BðSDÞ
and GðSDÞ
and SNr
xy ; respectively, inhibit GPi activities Bk
activities Gxy
; which, respectively, inhibit thalamic activities Vk and collicular activities Sxy : As detailed in Fig. 3, if the FEF saccade plan matches the most
salient sensory input to the PPC, then the BG disinhibit the SC to open the gate and generate the saccade. However, if there is conflict between the bottom-up
input to PPC and the top-down planned saccade from FEF, then the BG-SC gate is held shut by feedforward striatal inhibition (note BG blocks labeled GABA)
until the cortical competition resolves. When a plan is chosen, the resulting saccade-related FEF output signal Fxyi
activates PPC, the STN and the SC ðSxy Þ:
The SC excites FEF postsaccadic cell activities Fxy
; which delete the executed FEF plan activity. The STN activation helps prevent premature interruption of
plan execution by a subsequent plan or by stimuli engendered by the early part of movement.
1991; Crosson, 1985; Hikosaka & Wurtz, 1983; Mink, 1996;
Mink & Thach, 1993; Redgrave, Prescott, & Gurney, 1999).
BG outputs provide GABA-ergic inhibitory gating of their
target structures. In the primate saccadic circuit, cells in the
substantia nigra pars reticulata (SNr) tonically inhibit the SC
but pause briefly to allow the SC to generate a saccade
(Hikosaka & Wurtz, 1983, 1989). Lesions in this system can
release a ‘visual grasp reflex’ (Guitton, Buchtel, & Douglas,
1985), i.e. impulsive orienting to any visually salient object.
Ancient vertebrate genera, such as frogs, already had a welldeveloped BG system (Marin et al., 1998). Though lacking a
precise equivalent of the primate saccadic circuit, frogs can
selectively orient while ignoring distracters, but lesions of
the BG projection to the OT (SC homolog) impair a frog’s
ability to orient selectively (Ewert, Schurg-Pfeiffer, &
Schwippert, 1996).
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Thus BG gates help create a difference between physical
and motivational salience. Such gating enables an actor to
acquire reward for foveating a physically weak stimulus
(e.g. a dim and motionless predator) while ignoring a
physically strong stimulus. In most visual scenes, many
targets compete for foveation. If the saccadic gate is opened
before competition among stimuli resolves, the system may
foveate the most contrastive target, or may attempt to
foveate multiple targets simultaneously by averaging
ambiguous SC activity (Lee, Rohrer & Sparks, 1988;
Ottes, Van Gisbergen, & Eggermont, 1984). In the
proposed model, feedforward striatal inhibitory interneurons (Gernert, Hamann, Bennay, Loscher, & Richter,
2000; Koos & Tepper, 1999; Wilson et al., 1989) keep the
BG gate shut until competitive dynamics in posterior
parietal cortex (PPC) and the frontal eye field (FEF) have a
chance to select a unique saccade goal.
The FEF and SC are individually sufficient to generate
saccades (Deng et al., 1986; Schiller, True, & Conway,
1980). Yet in the normal animal, the SC is an important
common pathway for saccade generation, and focal SC
lesions result in transient impairment of all saccade types
(Schiller et al., 1980). Imaging studies (Sweeney et al.,
1996) have shown that the frontal cortex is more strongly
activated in more difficult oculomotor tasks, e.g. those
requiring memory-guided saccades or anti-saccades. Such
tasks engage elements of the frontal oculomotor system,
including the PFC, FEF, and supplementary eye fields (SEF,
an oculomotor area in dorsomedial frontal cortex, DMFC).
Lesions of these frontal areas suggest that they incorporate
distinct, modular contributions to oculomotor planning and
control. The frontal oculomotor areas add the ability to
use: head-centered or other non-motor-error coordinates
(Schiller, 1998; Schlag & Schlag-Rey, 1987; Schlag-Rey,
Amador, Sanchez, & Schlag, 1997); working memory
(Goldman-Rakic, 1987, 1995; Pierrot-Deseilligny, Israel,
Berthoz, Rivaud, & Gaymard, 1993); conjunctions of
features (Bichot & Schall, 1999); and internal sequencing
(Sommer & Tehovnik, 1999). Taken together, these data
suggest a hierarchy. Visual inputs to the SC dominate
reactive movements by default, but plans within the frontal
cortex can assume control of the SC when simple reactive
eye movements are insufficient (Fig. 3).
These principles, realized here as mechanisms in a
saccadic control model, should also apply to adaptive
control of manipulative and cognitive behaviors. The model
and results were briefly reported in Brown, Bullock, &
Grossberg (2000).
2. Methods
The model realizes fourteen major computational
hypotheses. These are presented verbally and diagrammatically to frame the subsequent mathematical specification.
Table 1 lists abbreviations to be used in reference to
neuroanatomical structures. Many of the assumptions are
shared with prior verbal formulations and computational
models (e.g. Albin et al., 1989; Berns & Sejnowski, 1998;
Brown, Bullock, & Grossberg, 1999; Bullock & Grossberg,
1991; Contreras-Vidal & Stelmach, 1995; Crosson, 1985;
Dominey, Arbib, & Joseph, 1995; Graybiel, 1998; Hikosaka
& Wurtz, 1983, 1989; Houk & Beiser, 1995; Mink, 1996;
Mink & Thach, 1993; Redgrave et al., 1999; Schultz, 1998;
Suri, Albani, & Glattfelder, 1997; Wichmann & DeLong,
1996; Wickens, 1997), although in virtually all cases the
present mathematical implementation of a postulate has
predictive implications that distinguish it from postulates in
prior models. It is noted below where these predictive
implications depart most significantly from prior models.
The new model is called TELOS, which is from the ancient
Greek telos for goal, end, or completion of a plan, but is also
an acronym for TElencephalic Laminar Objective Selector.
The BG and cerebral cortex together make up the
telencephalon, and the BG take inputs from, and help select
the outputs of, laminar oculomotor structures, notably the
SC and the FEF in the frontal cortex. Through concerted
action of these structures, the current behavioral objective—
e.g. a desired eye movement vector or maintained eye
fixation—is selected.
2.1. Forebrain circuits enable competitive
limited-capacity planning
The brain uses analog neuronal states to represent
patterned information that is distributed across multiple
cells, which have finite ranges of membrane potential, firing
rates, and synaptic weights. To use these finite ranges
effectively, the brain employs normalizing mechanisms,
notably mutual inhibition within on-center off-surround
networks whose neurons obey membrane equations. By
normalizing their activities, such networks enable relative
activity levels to represent patterned information and
thereby minimize noise and saturation effects (Grossberg,
1973, 1982). The same normalizing mechanisms can also
mediate competition among plans (Bullock & Rhodes,
2003; Grossberg, 1978a). Early modeling proposed how
such a competitive network could create a working memory
in which the relative priority (intended performance order)
among the plans constituting a forthcoming sequence is
represented by the relative activation levels (Boardman &
Bullock, 1991; Grossberg, 1978a,b) of the plan representations. There is now compelling evidence for both the
normalization principle (e.g. Basso & Wurtz, 1998; Cisek &
Kalaska, 2002; Pellizzer & Hedges, 2003) and the
prediction that relative activation level codes relative
priority in working memory (Averbeck, Chafee, Crowe, &
Georgopoulos, 2002). The limited number of distinct
activation levels that can be simultaneously represented
implies a limit on the number of simultaneously active
(prioritized) plans in working memory, consistent with
many experimental reports (cf. Cowan, 2001).
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Fig. 3. Cortical and striatal processes in location-specific gating of the SC by the BG. (A) When multiple stimuli exist as potential saccade goals, the
corresponding PPC representations specifically excite striatal SPNs (shown in the rectangle within the BG rectangle) and non-specifically (convergently) excite
feedforward inhibitory interneurons (labeled with a capital sigma), via corticostriatal projections. If more than one saccade plan is active, then striatal
feedforward inhibition from all active plans prevents any one plan from activating its corresponding striatal SPNs to open the BG gate. This is because the
pooled inhibitory input to each SPN can overwhelm the specific excitatory input. Therefore the SC is not released from inhibition from the SNr, and movement
is prevented while the conflicting cortical plan activity remains unresolved. (B) Targets compete in PPC via inhibitory interactions. When competition resolves
so that the movement plan is unambiguous, the PPC’s excitatory input to striatal SPNs eventually exceeds striatal feedforward inhibition, which wanes as
competing plans lose activation and stop convergent excitation of striatal inhibitory interneurons. The winning SPN’s discharge inhibits SNr (opens the
normally-closed BG gate), which disinhibits part of the SC map. (C) If the FEF plans a saccade goal that differs from the location of a strong visual stimulus, the
competing frontal and parietal activities collectively drive striatal feedforward inhibition to keep the BG gate shut until the conflict resolves. (D) As the frontal
cortex imposes its saccade goal on the parietal cortex, the competition between saccade goals resolves, and the BG gate opens to generate the unambiguous
saccade. Note: The absence of an icon for FEF activity in (B) indicates not that FEF would be inactive in case (B), but only that FEF contains no plan contrary
to PPC in case (B).
2.2. The basal ganglia contextually gate expression
of reactive behaviors or plans
Parallel channels of the BG embody normally-closed
gates that cooperate with the thalamus, the SC and the FEF
to withhold saccadic eye movements until a single reactive
movement or saccadic plan is selected by competitive
dynamics in the SC, FEF, and PPC. This gating hypothesis
explains four data sets. (1) GABAergic inhibitory projection
neurons of the two BG output nuclei, the GPi and SNr
(internal or medial segment of the globus pallidus, and
substantia nigra pars reticulata), exhibit tonic firing rates
that are sufficiently high to suppress strong activations of
recipient neurons in thalamus or SC. (2) Phasic reductions
from these high resting rates, or complete pauses, release,
and scale the rate of, behaviors controlled by thalamic,
collicular, and other targets of GPi or SNr outputs (Bullock
& Grossberg, 1991; Hikosaka & Wurtz, 1983, 1989; Horak
& Anderson, 1984; Skinner & Garcia-Rill, 1990; Takikawa,
Kawagoe, Itoh, Nakahara, & Hikosaka, 2002; Turner,
Grafton, Votaw, Delong, & Hoffman, 1998). (3) Lesions in
the BG pathway lead to a spectrum of disorders of voluntary
movement (Wichmann & DeLong, 2001; Young & Penney,
2001), ranging from a hypokinetic extreme—bradykinesia
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
hyperdirect BG pathways (e.g. Levy et al., 1997; Nambu,
Tokuno, & Takada, 2002).
Table 1
Anatomical abbreviations
Basal ganglia, including striatum,
GPi, GPe, STN, SNc, SNr
The frontal eye fields
GABA-ergic striatal interneurons
The external (lateral) segment of the
globus pallidus
The internal (medial) segment of the
globus pallidus
Anterior infero-temporal cortex
Posterior infero-temporal cortex
Lateral posterior/pulvinar nuclear
complex of the dorsal thalamus
Optic tectum (called the SC in
Prefrontal cortex
A pallidal- (GPi) or a nigral- (SNr)
receiving zone of the thalamus, e.g.
mediodorsal, ventral anterior, and
ventral lateral pars oralis nuclei
Posterior parietal cortex
The substantia nigra pars compacta
The substantia nigra pars reticulata
Spiny projection neuron of the dorsal
striatum (caudate or putamen)
The subthalamic nucleus
Ventral tegmental area
(abnormally slow movement) and loss of ability to initiate
planned movements—to a hyperkinetic extreme, including
saccadic hyperdistractibility, hemi-ballism, and chorea.
Thus different lesions produce too much or too little BG
inhibition of plan execution. (4) Pathway tracing and
physiological studies indicate an impressive degree of
independence among a large number of parallel circuits that
traverse the BG and return to a specific region of origin in
frontal cortex (Alexander & Crutcher, 1990; Middleton &
Strick, 2000). The BG thus contain a large set of parallel,
programmable, gates.
2.3. Activation of the BG direct pathway can release
plan execution
Within each parallel BG circuit, the direct pathway-the
monosynaptic pathway from the striatum to GPi or SNrenables transient opening of BG gates. A typical voluntary
behavior is released via the direct pathway when a small
set of striatal medium spiny projection neurons (SPNs)
become active and inhibit a set of GABAergic projection
neurons in the GPi or SNr (Figs. 2 and 3B). Thus, gate
opening works by inhibiting a tonically-on movement
inhibitor. Although this interpretation is supported by many
saccadic eye movement studies (e.g. Handel & Glimcher,
1999, 2000; Hikosaka & Wurtz, 1983, 1989), its generality
remains to be established, in part because of the complexity of interactions between the direct, indirect, and
2.4. Feedforward inhibition in the striatum mediates
competition for plan expression
The withholding function of the normally-closed BG
gates opposes tendencies to react immediately to whatever
stimulus is physically most salient, and creates time for
plans supported by motivational and cognitive information
to activate, compete, and be selected. The striatum of the
BG provides a competitive arena in which representations of
alternative actions vie for execution. Competition can be
mediated by surround inhibition, which could be recurrent
(feedback), feedforward, or a combination of the two.
Despite receiving massive numbers of excitatory fibers from
cortex, the striatum is known as a ‘silent structure’, in which
only a small percentage of the dominant neuron type, SPNs,
is strongly active at any one time. Prior proposals, that a
striatal choice-making competition is mediated primarily by
recurrent inhibitory collaterals of SPNs, cannot be logically
reconciled with a ‘silent’ striatum, because recurrent
competition would require significant supra-threshold
activation of many competing SPNs. Moreover, data
suggest that striatal recurrent inhibition is weak (Jaeger,
Kita, & Wilson, 1994). We therefore propose that cortical
plan representations bid in parallel for execution via
excitatory inputs to corresponding striatal SPNs, and oppose
execution of other plans by exciting GABAergic striatal
feedforward interneurons (GABA-SIs) that are known to
strongly inhibit striatal SPNs (Gernert et al., 2000; Koos &
Tepper, 1999; Wilson et al., 1989). Because the ratio of
GABA-SIs to SPNs is , 1:20, such competition by
feedforward surround inhibition is entirely consistent with
a ‘silent’ striatum.
2.5. Striatal activation requires convergent inputs
from distributed plan representations
Gate opening, for any cognitive/mnemonic or motoric
degree of freedom controlled by the BG system, is usually
withheld until a single plan for that degree of freedom
dominates alternative active plans. Examples of cases to
manage are: two parietal cortex bids (Fig. 3A), or one
parietal and one FEF bid (Fig. 3C), to move the eyes in
conflicting directions. Expression of two conflicting plans at
the same time would result in incoherent behavior. Evidence
that this problem does occur when gate opening is too rapid
comes from data on saccadic averaging: the probability of
mistakenly saccading to the average position, halfway
between two visual target loci, is a decreasing function of
reaction time (RT) and target separation (Ottes et al., 1984).
Evidence that the probability of trying to execute two plans
at once can be strongly affected by BG lesions comes from
studies of postural responses in PD (Parkinson’s Disease)
patients. Such patients exhibit non-adaptive ‘hybrid’
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
responses that appear to reflect execution of two plans that
are strict alternatives in healthy control subjects (Horak,
Shupert, & Mirka, 1989). In another BG disease syndrome,
HD (Huntington’s Disease), involuntary choreic movements, once thought to be random, often result from
repeatable phasic coactivations of normally antagonistic
muscles (J. Mink, personal communication, September
2002). Although tonic coactivation could arise by involuntary activation of a single normal behavior (e.g. Humphrey
& Reed, 1983), phasic coactivations would more likely arise
from involuntary co-execution of two (or more) plans that
are normally mutually exclusive.
Incoherent behavior can be avoided if broad feedforward
inhibition in the striatum is complemented by a high
threshold for activation of SPNs, so that they can be
activated only by strong convergent excitation from cortex.
Then SPNs will not activate until the several cortical
representations associated with a degree of freedom, which
are hypothesized to converge in the striatum, simultaneously project significant excitation to the same localized
striatal region. As competition between plans resolves in
favor of one winner, the cortical state becomes more
coherent, in the sense that the remaining highly active sites
are mutually compatible representations that help to specify
the winning plan. Simultaneously, the signal from cortex to
striatum becomes more focused, as diffuse signals from
losing plans decline and signals from the winning plan’s
distributed representations grow and converge to excite the
associated small region of striatum (Fig. 3D). This
hypothesis computationally clarifies reports that convergent
cortical afferents to distinct localized regions of the striatum
originate from cortical areas that are themselves strongly
interconnected (Cavada & Goldman-Rakic, 1991; Flaherty
& Graybiel, 1991; Gerfen & Wilson, 1996; Yeterian &
Hoesen, 1978). It is also consistent with the gap junctions
that may mediate broad activation of GABA-SIs (Kita,
Kosaka, & Heizmann, 1990), and SPNs’ specialization to
switch from OFF to ON states only if they receive strong
convergent cortical excitation (Wilson, 1995a).
2.6. Indirect pathway activation enables deferral
of a chosen plan
In ‘simple’ (as opposed to ‘choice’) RT tasks, an oftrewarded plan can be cognitively primed for execution, yet
await a permissive event that signals when it may be
released. During the priming interval, excitation of SPNs in
the direct pathway could inhibit the GPi or SNr and cause
premature execution unless such gate opening were stopped
until occurrence of the permissive event. The trainable ‘GO
signal’ function provided by the direct pathway is therefore
complemented by a trainable ‘STOP signal’ provided by the
indirect BG pathway. Thus the model distinguishes two
bases for non-execution of a plan. In one case, the plan may
fail to satisfy the conditions for winning the direct
pathway’s striatal competition. In another case, an action
that satisfies those conditions can be blocked by activity in
the indirect pathway. Evidence that activation in the indirect
pathway can be potent enough to serve this function comes
from studies of PD, in which hyperactivity of the indirect
pathway has been associated with inability to initiate a
planned action (Wichmann & DeLong, 1996). The striatal
source cells for the indirect pathway affect the GPi (or SNr)
via two routes: striatum-GPe-GPi (or SNr) (Hazrati, Parent,
Mitchell, & Haber, 1990; Parent & Hazrati, 1995; Smith &
Bolam, 1990), and striatum-GPe-STN-GPi (or SNr) (e.g.
Wichmann & DeLong, 1996). Consistent with recent
experimental results (Hassani, Moroux, & Feger, 1996;
Levy et al., 1997), this hypothesis differs from various prior
models by treating the former (shorter) route as the more
potent, and thus the more important for the STOP function.
Below, hypothesis 14 specifies a distinct function for the
STN-GPi link, which is part of the so-called ‘hyper-direct’
2.7. Thalamo-striatal feedback guides learning of indirect
pathway STOP responses
Learning necessary to generate a STOP signal occurs
during trials on which premature release of a movement
leads to non-reward. In the model, such experiences can
lead to learned activation of the indirect channel associated
with the released movement, even though striatal activation
in the indirect channel is not a normal part of releasing the
movement. How does the brain identify which indirect
pathway it should recruit in order to STOP a given direct
pathway? We propose that the feedback pathways (Fig. 2)
from the thalamus to the striatum (De Las Heras, Mengual,
Velayos, & Gimenez-Amaya, 1998; McFarland & Haber,
2000) help solve this credit-assignment problem by routing
a specific teaching signal to the indirect channel associated
with whatever direct channel has just activated. Because the
thalamo-striatal pathways have been neglected in prior
models of the BG, and there is little pertinent electrophysiological work, this hypothesis is currently based
largely on anatomical and computational considerations.
2.8. Laminar maps enable gated interactions between
planning and executive cells
Alternative plans are coded, in the SC and FEF, at
distinct spatial positions in 2-D cellular arrays or maps. To
complement the BGs parallel gating system, ‘planning’ cells
encode preparatory activities and send bids to the BG, and
associated ‘executive’ cells generate phasic outputs if and
when their BG gate opens. This hypothesis predicts a multilayered—i.e. laminar—organization and a distinctive pattern of connectivity. Such is true of the SC, which projects
to the BG (via dorsal thalamus, e.g. LP/PUL) from
superficial layer maps (e.g. Butler & Hodos, 1996; Hall &
Lee, 1993; Hutsler & Chalupa, 1991) and receives SNr
outputs in deeper layer maps that are representationally and
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
topographically ‘in-register’ with superficial layers. As
schematized in Fig. 2, there is also abundant anatomical
evidence for a similar stratification of cells and fiber systems
in the frontal cortex (Berendse, de Galis, & Groenewegen,
1992; Canteras, Shammah-Lagnado, Silva, & Ricardo,
1990; Gerfen & Wilson, 1996; Giuffrida, Volsi, Maugeri,
& Perciavalle, 1985; Jones, Coulter, Burton, & Porter, 1977;
Levesque, Charara, Gagnon, Parent, & Deschenes, 1996;
Royce & Bromley, 1984; Wilson, 1995a,b; Yeterian &
Pandya, 1994). A large portion of cortical fibers projecting
to the striatum emerge from small pyramidal cells of layers
II – III or III– Va (depending on species and cortical area).
Although layer Vb large pyramids send some collaterals to
BG (including STN and striatum; see hypothesis 14 below),
such “Brainstem projecting neurons make up a very small
proportion of the total population of cortico-striatal cells….
The vast majority of (cortico-striatal) neurons are more
superficially situated, smaller pyramidal neurons (Wilson,
1995a, p. 38)”, which also have cortico-cortical connections. There is accumulating physiological evidence (Bruce,
Goldberg, Bushnell, & Stanton, 1985; Iwabuchi & Kubota,
1998; Sawaguchi, 2001; Segraves & Goldberg, 1987;
Turner & DeLong, 2000) that the cortico-striatal cells in
layers III– Va have different task-related properties than the
large pyramidal cells that are clustered in layer Vb and send
outputs to the brainstem and effectors. For the part of FEF
(area 8) that they examined, Iwabuchi and Kubota (1998)
found that neurons with early onsets coupled to a GO cue
(signaling whether to release a response) were numerous in
layer III, whereas neurons whose onsets were late and
movement-coupled were more numerous in layer V. Thus
there was a normal sequence involving activation of layer
III neurons before layer V output neurons. These data
support the hypothesis of at least distinct, if not completely
segregated (Sommer & Wurtz, 2000; Wurtz, Sommer, Pare,
& Ferraina, 2001) laminar distributions of ‘planning’ and
‘executive’ cells within areas of frontal cortex. Some
models assume or imply that such a functional differentiation is achieved across cortical areas, rather than by layers
within areas. However, moving across areas typically
implies a change in cortical representation. Columnar
laminar neocortex can efficiently implement the planning/
execution distinction across layers.
Although the concept of planning has some connotations
that go beyond the competence of the current model, the
current usage of ‘plan cell’ is consistent with prior usage
(e.g. Bullock & Grossberg, 1991). A neural representation
qualifies as a plan if it helps specify a possible forthcoming
response, and if the circuit controlling the effects of the
representation’s activation provides a basis for withholding
the active representation’s access to the effector apparatus
until a decision has been reached to execute the plan. To this
the current analysis adds another condition: that the putative
plan representations must send bids to the telencephalic
decision centers that gate their access to effectors. Thus the
response representations that exist in ungated reflex systems
do not qualify as plans; nor do gatable output stages that
send no projections to telencephalic decision centers.
Finally, although the plan cells in the current model can
remain active across significant delays after stimulus offset
(hypothesis 12 and Fig. 9F and G, below) this may not be
true of all cells properly so-called.
2.9. Small BG gating channels control large topographic
zones in laminar target structures
How broad a region of frontal cortex (or SC) is affected
by opening of a particular BG gate? On gross anatomical
grounds, the cortico-striatal-GPi/SNr projection has been
described as a ‘funnel’, and indeed there is an order of
magnitude reduction in numbers of cells from the pool of
cortico-striatal cells to the pool of striatal SPNs (Zheng &
Wilson, 2002), and another order of magnitude reduction
from the pool of SPNs to the pool of GPi/SNr projection
neurons (Oorschot, 1996; Wickens, 1997). The ‘funnel’
usage has waned because it was interpreted to imply mixing
rather than the (now well-established) segregation of
channels, but the radical reduction in cell counts implied
by ‘funnel’ remains a key constraint on models. In frontal
cortex, many more cells are reached by the projection from
PNR-THAL (pallidum or nigra receiving thalamus) than
there are cells in PNR-THAL, or in the GPi/SNr. We thus
hypothesize a one-to-many relationship between cells in
GPi/SNr and gatable cells in frontal cortex. It is probable
that, similar to SNr – SC interactions (Handel & Glimcher,
1999, 2000; Hikosaka & Wurtz, 1983, 1989), pausing by
each small cluster of cells in the GPi/SNr directly affects
(via PNR-THAL) a much larger set of cells that are
distributed across a moderate-sized zone of a given frontal
cortical area. Further cells will then be affected by
secondary, e.g. cortico-cortical, interactions.
Thus action-selection theories of the BG cannot plausibly
assume a one-to-one relationship between plan representations and BG channels. Models must explain how the
effect of gating can be highly selective and accurate even
though the number of cellular degrees of freedom in the
GPi/SNr is relatively small. In many cases, two or more
cortical plans will have similarly high activation levels, and
the gating signal needs to select the cortical plan that has
won the striatal competition rather than any highly active
alternatives. The current model hypothesizes that loops
through the BG circuit respect fronto-cortical topography:
each frontal zone receives thalamo-cortical fibers from a
zone of PNR-THAL that receives fibers from a zone of
GPi/SNr that receives fibers from the striatal zone targeted
by the frontal zone of origin. This principle has been
established for the large-scale topographic organization of
frontal cortex. For example, there are separate motor (area
4), premotor (area 6), oculomotor (FEF) and prefrontal
circuits (Alexander & Crutcher, 1990; Middleton & Strick,
2000). To explain how the system achieves sufficient
selectivity, the model proposes that such circuits also
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
respect finer-grained topographic organization. In particular, selectivity can be achieved even if the cortical activation
levels of two plans are indistinguishable, so long as the
plans exist in different gateable cortical zones (GCZs). For
simplicity, the model (see Fig. 5) defines three BG channels
and three associated GCZs in FEF, but these discrete
elements could be replaced by an overlapping continuum in
a straightforward way in a population model. The main
computational requirement is that the cortical activity foci
representing two competing plans be sufficiently separated
that one receives a reliably stronger GO signal from the
Hypotheses 4, 5 and 9 imply a kind of complementarity
in the macrocircuit’s management of competitive plan
execution. Any two cells that represent different plans in the
same GCZ are likely to compete via strong mutual intracortical inhibition, so that when the competition resolves
and the gate opens for the entire GCZ, only the winning plan
will execute. If two plan cells pertinent to a given degree of
freedom do not reside in the same GCZ, then they are less
likely to compete via such strong mutual intra-cortical
inhibition, so a large activation of one may not imply a
small activation of the other. However, in this case, striatal
competition can ensure that only one of the GCZs receives a
strong GO signal. In either case, a non-viable ‘hybrid’
response is generally avoided.
2.10. Selective gating of premotor zones enables fast
sensory-motor remapping
A gating system can control which of several alternative
sensory-motor or cognitive-motor pathways are used to
generate behavior. This ability is fundamental because a
visual stimulus may serve either as a movement target or as
a discriminative cue to move elsewhere. Sensory/cognitive
and motor information is often represented in different
coordinate systems. The model shows how learned
associations can form between such different coordinate
representations, notably from spatially invariant sensory/cognitive representations to spatially coded motor representations. For example, in the SC map, plan representations
cluster on the basis of eye movement direction and
amplitude (a vector code), not on the basis of the stimulus
modality (vision, audition, or touch) that may excite them.
In contrast, premotor frontal cortex embodies superimposed
gradients, across the cortical map, based on modality
and source of cortico-cortical input. This creates a multiplexed patchiness in receptive field structure and the
sensitivity of local cortical zones to perceptual, categorical,
and mnemonic inputs. Within each patch, many movement
vectors are also represented, so such movement features are
not the basis of map structure. In particular, in premotor
cortex (area 6), there is a well-established medio-lateral
gradient of sensitivity to proprioceptive vs. exteroceptive
inputs (Mushiake, Inase, & Tanji, 1991; Passingham, 1993).
In FEF, which spans major parts of areas 8 and 45, there is
a medio-lateral gradient for saccadic amplitude, from largeto-small saccades, in the rostral bank of the arcuate sulcus
(Bruce et al., 1985). Moreover, this medio-lateral gradient
for amplitude reflects differences in the nature of the
afferents arriving in subareas of the FEF (Barbas, 1988;
Barbas & Mesulam, 1981; Bullier, Schall, & Morel, 1996;
Schall, Hanes, Thompson, & King, 1995b; Schall, Morel,
King, & Bullier, 1995a). The medial, ‘large-saccade’, part
of FEF receives afferents from dorsal stream visual and
auditory areas that process information about stimuli in, or
even beyond, the visual periphery. This medial FEF area
appears to lack significant inputs from ventral stream areas
TEO/V4 (ITp) and TE (ITa), both of which preferentially
represent stimuli in the foveal and parafoveal regions of the
visual field. In the lateral FEF, a strong projection from ITp
terminates in FEF zone 45A and a strong projection from
ITa terminates in FEF zone 45B (Bullier et al., 1996). Thus
FEF zones receive functionally distinct types of inputs.
The model’s GCZs in FEF reflect these input differences.
Specifically, the model incorporates the fact that cells in ITp
are sensitive to simple features falling within particular
retinotopic loci (Kobatake & Tanaka, 1994; Tanaka et al.,
1991; Komatsu & Ideura, 1993), whereas ‘position
invariant’ cells in ITa are sensitive to feature complexes
(‘objects’) regardless of specific retinotopic locus (Gross,
Desimone, Albright, & Schwartz, 1985; Tanaka et al.,
1991). These systematic differences in the types of cues to
which the FEF sub-areas are sensitive are consistent with
the more general pattern observed for motor and premotor
areas of frontal cortex (Passingham, 1993). The model
proposes that sensitivity to distinct classes of inputs covaries
with the GCZs. Thus, unlike BG gating of SC, BG gating of
cortex operates on plan clusters defined by cue type and
modality. The emphasis on cue sensitivity differences in
FEF may seem surprising, because it was established long
ago that FEF neurons may show no selectivity for stimulus
properties such as shape or color (e.g. Mohler, Goldberg, &
Wurtz, 1973). However, evidence on the anatomical
projections from feature sensitive areas such as ITp, when
combined with physiological evidence on the emergence of
feature selectivity in FEF neurons when features are
consistently rewarded (Bichot, Schall, & Thompson,
1996), support the present hypothesis, as well as hypothesis
12 (below) regarding reward-guided learning mediated by
weight changes on the IT to FEF pathways.
Fig. 5A illustrates this hypothesis. Maps of retinotopic
inputs and motor error outputs at cortical and collicular levels
are modeled by 3 £ 3 cell grids. Each square within a grid
represents a model cell with a receptive or movement field
corresponding to its grid position, so each grid represents
fixation (the null direction) plus eight saccade directions
(each with unit amplitude). Fig. 5B shows that each of the
three FEF zones has input, plan, and output layers.
Overall, the model has twelve gateable zones, three
cortical zones of 9-cells each, and nine collicular zones of
1-cell each. The model approximates the SNr –SC system
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
with a map-like array of nine SNr cells, each with output
restricted to a corresponding region of the 9-cell SC map.
Each SNr output to the SC is specific for a zone coding
similar saccadic directions and amplitudes (Bayer, Handel,
& Glimcher, 1999; Hikosaka & Wurtz, 1989). Thus, there is
not a single non-specific release of the entire SC when a BG
gate opens. Although gating is specific to a subset of the SC
map, it is coarse relative to the direction-specifying signals
that are received by the SC from other structures, such as the
PPC, FEF, and the retina (which is not modeled). Thus,
saccadic accuracy depends mostly on non-nigral inputs to
the SC, but saccadic occurrence, and to a lesser extent,
velocity, depend on the degree of release of nigral inhibition
(Takikawa et al., 2002). Greater realism could be achieved
by scaling up the resolution of the SC map relative to the
SNr map, but this was unnecessary for present purposes.
For the model FEF, there are three BG gates and
corresponding PNR-THAL cells, each used to excite the
entire grid within one of the three model GCZs. These GCZs
correspond to patches of FEF that differ in whether they
receive inputs from ITa or ITp, and, for the two GCZs with
inputs from ITp, whether they respond more strongly to one
discriminative visual feature or another. Thus each FEF
GCZ controls a full range of saccadic directions, but its cells
have a similar sensitivity to one or another visual feature,
e.g. ‘red’ or ‘T-shaped’. In the following exposition, one of
these GCZs is called the GCZf (GCZ, fixation feature)
because cells in its ITp receptive field respond selectively to
salient features of the fixation stimulus that distinguish it
from the target stimuli to which the monkey will saccade
after fixation light offset (Fig. 1). The second GCZ is called
the GCZt (GCZ, target feature) because its ITp receptive
field is selective for salient features of target stimuli that
distinguish them from the fixation stimulus. The model’s
third FEF gateable zone (Fig. 5B) is called the GCZo
because it corresponds to the part of area 45 that receives
inputs from ITa (area TE) cells that show selectivity for
feature complexes (‘objects’) regardless of where these
complexes fall on the retina. Although the GCZo does not
receive a retinotopic projection from ITp, it does have other
inputs, from the SC and LIP (not shown in Fig. 5), that
endow it with retinotopically organized planning and output
layers. These two properties allows the GCZo to learn to
associate a spatially invariant object category with a saccade
direction, thus mapping a What stream sensory cue to a
Where stream saccade plan. These GCZo cells thus mediate
a learned What-to-Where transformation between different
coordinate systems. Although cells in the GCZf and GCZt
are in the What stream, in a sense they mediate a Where-toWhere transformation because their feature-specific filtering also reflects retinotopic organization within ITp.
With these three GCZs, the model is able to open one
gate in the presence of two or more competing visual
stimulus representations, and release movement only
toward the stimulus that matches the feature or object
class associated with the open gate. Thus, the correct
movement will occur provided that the plan cell sitting in
the correct GCZ becomes able (via reinforcement learning,
see below) to win the striatal competition to open its gate.
In a larger scale implementation, these fully discrete
channels would be replaced by overlapping populations.
2.11. Sustained plan cell activity is a local cortical circuit
property modulated by layer VI
Cortical cells whose activity correlates with plans often
show task-dependent sustained firing that outlasts the
stimulus presentations that cue the initial activation of
these cells. In the model, sustained activation results from
local, intra-cortical, recurrent excitation whenever the plan
cell resides in a GCZ that has been aroused or primed. Such
activation occurs when input from PFC reaches plan cells
via a cellular link in cortical layer VI; see Fig. 2. These same
layer VI cells are a source of excitation to the PNR-THAL
cells whose disinhibition allows plans to execute. Differential priming via layer VI allows the frontal cortex to bias the
pool of candidate plans to favor those that fall under the
particular input –output relation enabled by the primed GCZ
(Asaad, Rainer, & Miller, 2000; Hoshi, Shima, & Tanji,
1998). In contrast, some models have proposed that the basis
of sustained frontal activations is a cortico-thalamo-cortical
loop that starts with a cortical plan cell, traverses a thalamic
stage subject to inhibition by the GPi or SNr, and returns to
the plan cell of origin. In such a model, a BG gate must open
to enable sustained activity of each plan cell. The latter
seems unnecessarily restrictive for brief maintenance of
candidate plans in FEF before one plan is selected and
released, and may not even be necessary for prolonged
storage in PFC (cf. Durstewitz, Seamans, & Sejnowski,
2000). Moreover, anatomical studies (cited in Table 2) show
that the thalamo-cortical projection (to layers III and V)
from PNR-THAL may not even reach, and is certainly
not restricted to, the layer VI cells of origin of
the cortico-thalamic projection. Thus, there seems to be
no anatomical basis for restricting modeled thalamo-cortical
input so that it excites only plan cells, and therefore no basis
for preventing this input from activating output cells in layer
Vb. If so, it cannot serve the priming function proposed in
alternative models.
2.12. Reinforcement signals guide learning
of cortico-cortical and cortico-striatal links
Reinforcement learning can modify links from representations of arbitrary contexts and cues to plan representations. It can also affect interactions among candidate
plans. Phasic, non-specific, dopaminergic signals emerging
from the VTA/SNc in response to unpredicted rewarding
events (Schultz, 1998) reach both frontal cortex and the
striatum, where they modulate synaptic plasticity (Bao,
Chan, & Merzenich, 2001; Berger, Trottier, Verney, Gaspar,
& Alvarez, 1988; Calabresi et al., 2000; Charpier & Deniau,
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Table 2
Selected anatomical studies
Layer VI
Distant cortex
Layer III
Layer III, V
Layer III, V
Layer II, III
Layer V
Rhesus monkey, rat
Layer VI
VL (Thalamus)
Layer V
VA, MD nuclei of thalamus
Layer V, III
GPe, GPi
Rhesus monkey, rat
Squirrel monkey, rhesus monkey
Squirrel monkey
SC, intermediate layers
FEF (via thalamus)
FEF zone 45B
FEF zone 45A
Cebus monkey
Kang and Kayano (1994)
Iriki et al. (1991)
Kimura, Caria, Melis, and Asanuma (1994)
Jones et al. (1977), Levesque et al. (1996),
Yeterian and Pandya (1994)
Yeterian and Pandya (1994), Levesque et al. (1996)
Zin-Ka-Ieu et al. (1998)
Canteras et al. (1990), Gerfen (1992)
Hazrati et al. (1990), Parent and Hazrati (1993, 1995)
Smith and Bolam (1990)
Parent and Hazrati (1993), Shink, Bevan,
Bolam, & Smith (1996)
Lynch et al. (1994), Sommer and Wurtz (1998)
Bullier et al. (1996)
Bullier et al. (1996)
1997; Cohen, Braver, & Brown, 2002; Eyny & Horvitz,
2003; Gurden, Takita, & Jay, 2000; Law-Tho, Desce, &
Crepel, 1995; Reynolds, Hyland, & Wickens, 2001;
Wickens, Begg, & Arbuthnott, 1996). In the model,
arbitrary cueing of plan activation is learned through
reinforcement-guided modification of cortico-cortical links
terminating in frontal cortex; see Fig. 2. Then these cueactivated plan representations compete while exciting
GABA-SIs and SPNs in the striatum. Learned weights on
the cortico-striatal projections to SPNs can affect the
competition for plan execution and thereby facilitate
appropriate queuing of plans that are rewarded if performed
in the proper context/sequence, but not otherwise. Consistent with recent data (Zheng & Wilson, 2002), this proposal
departs from views of the striatum as a general adaptive
pattern recognizer capable of representing arbitrary combinations of cortical states (Amos, 2000; Graybiel, 1998). In
our model’s BG direct and indirect channels (see Fig. 4),
phasic dopaminergic bursts and dips (Brown et al., 1999;
Schultz, 1998) have opposite learning effects (Eyny &
Horvitz, 2003), mediated by the differential expression of
D1 and D2 dopamine receptors in striatal SPNs of the two
channels. In particular, DA bursts cause increments of
weights on recently active pathways to D1, substance P (SP)
and dynorphin (DYN) expressing SPNs of the direct
channel, whereas DA dips cause increments of weights on
recently active pathways to D2 and enkephalin (ENK)
expressing SPNs of the indirect channel. This is consistent
with observations (review in Steiner & Gerfen, 1998) that
D1 and D2 receptor activations normally exert opposite
effects on IEG (immediate-early gene) expression in SPNs.
D1 activation induces IEG expression in D1 SPNs, but D2
activation does not induce IEG expression in D2 SPNs.
Instead, given cortical input or NMDA receptor activation,
D2 blockade induces IEG expression in D2 SPNs. Note that
the D2 blockade may be equivalent to what occurs naturally
when DA input dips due to pausing of SNc cell activity
(Schultz, 1998). Overall, we emphasize cortico-cortical
learning for control of the direct channel and cortico-striatal
learning for control of the indirect channel. The latter is
supported by recent data of Berretta, Parthasarathy, and
Graybiel (1997). Unlike some (e.g. Young & Penney, 2001),
we do not interpret the Berretta et al. (1997) data to imply a
scarcity of adaptive mono-synaptic cortico-striatal links to
the direct pathway. Many other data, e.g. the nature of
cortical and SNc axon synapses with spines on virtually all
SPNs, and the IEG and plasticity studies cited above, argue
The model predicts that dopaminergic (DA-ergic)
reinforcement signals interact with traces of stimulus and
plan activations, not with plan activations, or responses, as
such. When a delayed reinforcement signal signifies
Fig. 4. Direct and indirect pathways in the BG circuit model. The BG direct
pathway projects from the striatum to the GPi or SNr. Because it disinhibits
thalamo-cortical excitation, activation of the direct pathway normally
generates a ‘GO signal’ that enables action plans to be executed. Activation
of the indirect pathway, which projects from the striatum to the GPe to the
GPi or SNr, inhibits PNR-THAL and can thus STOP planned actions, even
if there is activation in the direct pathway that would otherwise suffice to
generate a GO signal. The feedback from thalamus to striatum helps solve a
credit assignment problem inherent in learning stimulus control of STOP
signals (see text for details). Weights on cortico-striatal afferents of SPNs
are subject to learning that depends partly on dopaminergic (DA) signals. In
the indirect pathway, a phasic DA dip disinhibits D2 receptors and permits
LTP. In the direct pathway, a DA burst rather than a dip permits LTP by
activating D1 receptors. Model cortico-striatal synapses on SPNs also
exhibit LTD, consistent with Calabresi et al. (2000).
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
a positive outcome, it can strengthen an input’s ability to
activate the plan representation causally responsible for the
rewarded act at a time when the plan representation has
already been de-activated by response feedback. In the
model, the time between termination of plan cell activation
and the reinforcement signal is assumed to be bridged by an
intracellular second messenger, which provides a short-term
‘trace’ of the earlier plan activation. Such traces are
predicted to be of brief duration, consistent with data
showing that reinforcement learning falls off steeply when
reward occurs more than a few seconds after the response.
The time scale of these traces is shorter than that of working
memory traces in dorsolateral PFC, which is not needed for
reinforcement-guided learning or performance of externally
cued behavior (Goldman-Rakic, 1987).
2.13. Feedback from superior colliculus to FEF serves
learning and performance functions
STN (Parent & Hazrati, 1995). Further components of
lockout protection may be supplied by specialized membrane properties (Wilson, 1995a) and recurrent GABAergic
collaterals of SPNs, but these have not yet been incorporated
in the model. Although hypothesis 14 differs from the recent
proposal regarding the ‘hyperdirect’ pathway made by
Nambu et al. (2002), their proposal was based on electrical
stimulation studies that did not take account of either the
laminar differences between (most) cortico-striatal and
cortico-STN cells or the physiological differences between
such cells, as suggested by the results of Iwabuchi and
Kubota (1998) and explicitly probed by Turner and DeLong
The macrocircuit implied by these fourteen hypotheses
is schematized in Fig. 2, which also serves as a key to
most of the important variable names in the
mathematical specification of the model. Table 2 tabulates
anatomical studies supportive of the connectivity shown
in Fig. 2.
Whenever any collicular zone becomes active enough to
exceed a threshold, it excites FEF plan layer cells tuned to
the same direction and grid position, across the three GCZs.
This excitation of FEF plan cells by SC (via thalamus;
Sommer & Wurtz, 1998) serves as a teaching signal that
enables GCZs not initially responsible for SC activation to
learn to take over responsibility for activating a specific SC
map region on future occasions. This teaching signal is
specific, and therefore more like the specific thalamostriatal teaching signal proposed in hypothesis 7 than the
non-specific DA teaching signal proposed in hypothesis 12.
When the collicular activation becomes large enough to
initiate a saccade, the SC to FEF excitation also activates
FEF inhibitory interneurons that can selectively ‘reset’ FEF
plan cells tuned to the same direction as the activated
SC cells. The model interprets the post-saccadic cells (see
Fig. 9E) found in FEF (Bruce et al., 1985) to be inhibitory
reset neurons (Fig. 2) that terminate specific plan representations once plan execution has initiated.
2.14.1. Mathematical model
Mathematical specification of the TELOS model’s
hypotheses required a system of differential equations,
whose exposition follows. The Results section may be read
before studying these equations. Figs. 6a and b depict every
model cell type, together with labeled inputs. These labels
refer to variables in the equations. Each cell type obeys a
nonlinear membrane (or ‘shunting’) equation (Grossberg,
1973; Hodgkin, 1964), in which excitatory and inhibitory
inputs affect separate conductances and thus do not act as
additive injected currents that directly affect activation/
potential. Often, a second and third equation further define
the membrane equation by computing the cell’s net
excitatory and inhibitory inputs, symbolized by superscripts
(E) and (I). In addition to explaining the form of the
equations, this section cites additional data that support
model details.
2.14. Cortico-subthalamic signals enhance lockout
of competing plans during plan execution
3. Visual inputs to the cortical cell types
Once a degree of freedom has been allocated to execute a
plan, it should be protected from further plan bids and from
interfering reactive movements (cf. Mink, 1996) while that
plan is executing. The basis for such protection in the model
is distributed, and includes the persistence of plan cell
activity into the movement interval. One anatomical basis
for locking out disruptive bids during execution is excitation
of the BG’s ‘hyperdirect’ pathway (cortex-STN-GPi) by
collaterals of the executive pathways. This hypothesis
explains observations that collaterals of frontal cortical
layer Vb projection neurons excite the STN (Canteras et al.,
1990; Gerfen & Wilson, 1996), which in turn excites the GPe
and GPi or SNr. It also explains why the GPi/SNr receives
focused inhibition from striatum but broad excitation from
External visual stimuli Ixyj
were convolved with a
Gaussian kernel to approximate visual cortical receptive
field properties, in order to generate the pre-processed
internal signal Ixyj (Fig. 2). Specifically,
Ixyj ¼
2ðp 2 xÞ2 2 ðq 2 yÞ2
where C is the set of eight nearest neighbors in the Cartesian
input space.
In the position-sensitive GCZs of the FEF, the signal Ixyj
then generated two further signals, namely Ixy
Ixyj (Fig. 2). The positional FEF input Ixy is a transient,
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
rapid-onset, input to FEF:
if 50 ms # t 2 txyj
# 80 ms
j xyj
This models the property that FEF input cells respond
quickly (with about 50 ms latency) to stimuli in their
retinotopic receptive fields, regardless of stimulus features.
The symbol txyj
is the time at which the convolved stimulus
signal Ixyj rises above 0.3, an arbitrary threshold chosen to
register activity but ignore spurious noise, and t is the
current simulation time. Eq. (2) specifies that input Ixy
FEF spans the interval of 50– 80 ms after Ixyj rises above
threshold. The interval was set to accord with data on FEF
cell onset and offset; see first burst in Fig. 9A. By default,
¼ 0 outside that interval, or if no inputs are active. The
quantity txyj
is not reset unless Ixyj first falls below the
threshold 0.3. If the features possessed by the external
stimulus are preferred by an FEF cell, then it fires a second
burst at around 100 ms. This is modeled with the stimulusðdÞ
discriminating FEF input signal, Ixyj
; assumed to arrive
from visual area V4 and ITp:
< Ixyj if 100 ms # t 2 tðonÞ # 130 ms
The interval 100 –130 ms. was based on timing of
the onset of the second burst in the data; see second burst
in Fig. 9A.
Input I(pc)
xy to the PPC is similar to the positional Ixy input
to FEF, except that there is no upper time limit, because PPC
activity is sustained as long as the visual input remains on
(Fig. 10C). Thus,
if t 2 txyj
. 50 ms
j xyj
Ixy ¼
The 50 ms onset delay reflects the time required for light
impinging on the retina to activate PPC, as seen in Fig. 11C.
In addition to these positionally-sensitive visual signals,
the model also invokes signals to the gated cortical zones
that respond to objects independent of their positions. Thus
inputs Tj from ITa cells (Figs. 2 and 6a) respond to visual
features of type j regardless of their spatial position (hence
the absence of a subscript xy). The dynamics of spaceinvariant anterior IT cells Tj were modeled by:
T ¼ 150ð1 2 Tj ÞIjðITÞ 2 30Tj ;
dt j
Thus Tj can be excited by model input IjðITÞ to a maximum
value of 1, and can also spontaneously decay.
4. Reactive and attentive processing: PPC, SC,
and SNr gating signal
The model supposes that developmental processes have
enabled the FEF representations of oculomotor plans to be in
register with the SC and PPC representations; see Gancarz
and Grossberg (1999) for a consistent model that proposes
how this happens.
PPC cell activities Pxy (Figs. 2 and 6a) represent
responses in the lateral bank of the intraparietal cortex
(LIP), which code visual stimuli in motor error coordinates
(Gnadt & Andersen, 1988). These activities were
Fig. 5. Grid of gatable zones and laminar organization. (A) Maps of retinotopic inputs and motor error outputs at cortical and collicular levels are represented
by two-dimensional grids. For convenience, the cells are indexed by a Cartesian coordinate system, which could be replaced by a space-variant representation.
(B) The model FEF contains three distinct gatable cortical zones (GCZs), each possessing input, plan, and output layers, as well as some distinctive cortical
afferents. These three GCZs are indexed by i, whereas k (see Fig. 2) indexes the corresponding three BG-thalamic channels that control them.
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
cells was defined by
ðPÞ þ 4
f ðFxyi ÞÞ þ 2ð½Fxyi
Þ ;
xy ¼ 5ðIxy þ
modeled by:
P ¼ 25½ð1 2 Pxy ÞPðEÞ
xy 2 Pxy Pxy :
dt xy
Eq. (6) says that Pxy can be excited by net input PðEÞ
xy to a
maximum value of 1, and inhibited by net input PðIÞ
xy to a
minimum value of zero. The excitatory input PðEÞ
which includes excitation from visual signals (Ixy
; defined
below in (8)), FEF output layer cells (Fxyi ; defined in (21)),
and FEF plan layer cells (Fxyi
; defined in (14)). Here the
summation index i ranges from 1 to 3 to represent different
FEF zones, as in Fig. 5. The notation ½wþ ¼ maxðw; 0Þ
Fig. 6. Modeled afferent signals, and mathematical symbols for physiological variables, associated with model cell types. As implied by Fig. 2, the model
includes a large number of cell types distinguishable by their afferent and efferent connections. See text for details. (a) Cell types of the cortical model. (b) Cell
types of the subcortical model.
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Fig. 6 (continued )
means that the argument w is thresholded at zero. The visual
signals to PPC can drive reactive saccades. However, the
potent input from FEF can override the visual signals and
reprogram the PPC to reflect the spatial goal of a previously
rewarded saccadic plan.
For reactive saccades, the strengths of competing visual
signals must be unequal for a decision to be made. Both
cortical magnification and motivational signals play a role
in the competition that determines PPC attentional focusing
(Platt & Glimcher, 1999). Here cortical magnification
sufficed, and the visual inputs Ixy
to the model PPC cells
were defined by:
¼ Ixy
þ 0:01ð1:1lx 2 1l þ ly 2 1lÞ:
This ensures that given multiple inputs of equal
strength, peripheral inputs are more likely to evoke a
saccade than foveal inputs, and objects left or right of
the fovea are more likely to attract the eye than objects
above or below. Nonetheless, this peripheral bias is not
sufficient to maladaptively break fixation of a visible
The inhibitory input PðIÞ
xy to PPC cells in (6) was
defined by
ð½Ppq þ Þ4 :
xy ¼ 1 þ 10Pxy þ 200
Term PðRÞ
xy specifies recurrent self-inhibition, which
ensures phasic PPC activity, as seen in Fig. 11C. Dynamics
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
for the interneuron mediating self-inhibition were defined by
d ðRÞ
P ¼ 2½ð1 2 PðRÞ
xy ÞPxy 2 Pxy :
dt xy
The summation term in (9) describes quickly-acting
lateral inhibition.
Activities Sxy of SC cells (Figs. 2 and 6b) were excited by
signals from PPC (Pxy) and the FEF output layer ðFxyi
Þ; and
inhibited by the topographic and potent SNr output, GðSNrÞ
xy ;
and a tonic signal set to a value of 10:
Sxy ¼ ð1 2 Sxy Þð60Pxy þ 5 f ðOÞ ðFxyi
2 0:3þ þ 10Þ
2 Sxy ð800½GðSNrÞ
d ðGÞ
Fi ¼ ð1 2 FiðGÞ Þð80Tj þ 2000 WjiðCGÞ ½Cj 2 0:35þ Þ
2 160FiðGÞ :
from SNr (defined in (32))
The inhibitory input
tends towards 1 when the BG gate is shut, and towards zero
when open. In the former case, Eq. (11) implies that SC
activity will be low even in the presence of strong excitatory
inputs. When the gate opens as GðSNrÞ
is driven near zero, Sxy
can become highly active in response to inputs from PPC or
FEF. In the simulations, the SNr output signal GðSNrÞ
had the
same indices and spatial resolution as the model SC map.
Although the in vivo SC map has much higher resolution
than the SNr output, this higher resolution was not needed
for the data simulated. Also, the simple SC model used here
has SC burst cells but omits SC buildup cells (Munoz &
Wurtz, 1995). A consistent but more detailed SC model,
with buildup cells as well as burst cells, was described in
Grossberg, Roberts, Aguilar, and Bullock (1997), but the
added detail was not needed for the data modeled in this
article. Because of its simplified SC treatment, the present
model does not include connections from FEF plan cells to
the SC, a projection that is consistent with the computational hypotheses of the model and that appears to be
justified by physiological observations (e.g. Sommer &
Wurtz, 2000).
5. Planning in the frontal eye fields
The activity Fxyi
of an FEF input cell (Figs. 2 and 6a) was
defined by:
d ðIÞ
F ¼ 60ð1 2 Fxyi
þ Ixy
2 Fxyi
dt xyi
þ 30
ðIrsðpÞ þ Irsj
(12)) among FEF visual cells (Schall et al., 1995b) helps to
normalize the overall activity and allows a stronger response
to a unique, isolated stimulus than to multiple stimuli,
consistent with the ‘oddball discrimination’ effect observed
in the FEF (Thompson, Bichot, & Schall, 1997).
The model FEF layer VI cells arouse a zone of cells in
the columns above (see Figs. 2 and 6a). Model layer VI
activity FiðGÞ was defined by
Here the index i in the subscript of Fxyi
ranges from 1 to 3
and denotes FEF GCZ, as in Fig. 5. These model FEF
‘visual cells’ (Schall et al., 1995a) respond to excitation
from areas including V4 and ITp, and are predicted to reside
in granular or supragranular layers. The excitatory inputs
(where j denotes stimulus feature) and Ixy
were defined
in (2) and (3). Feedforward lateral inhibition (the term in
Input Tj from ITa provides a training signal that
enables positionally-invariant inputs Cj ; from prefrontal
working memory cells to learn to select a zone of FEF
cells, constituting a premotor zone i; by activating
module i’s layer VI cells. The adaptive weight WjiðCGÞ ;
on the path from PFC site j to FEF module i, denotes a
synaptic strength that can be modified by reinforcement
learning; see Eq. (55).
Model FEF plan layer cells correspond anatomically to
layer III/Va cells from which the cortico-striatal projection
arises (Figs. 2 and 6a), and physiologically to FEF ‘visuomovement cells’. As shown in Fig. 5, they may be fixationor saccade-related, and they may perform a ‘What – Where’
or a ‘Where – Where’ transformation via distinct cortical
zones. The combination of these attributes leads to four
kinds of plan layer cells with distinct connectivity patterns.
Within a cortical zone, cells generally compete, but there is
no general between-zone competition. However, learned
selectivity of zone activation was found to develop more
quickly if the same saccade vector (including the special
case of fixation) does not have simultaneous strongly active
representations in different zones. This rate effect occurs
because weight change is a function of activation level.
Although not necessary for learning as such, the learning
speed advantage warrants the hypothesis that competition
exists between like vectors in different zones. Moreover, a
basis exists in the model for self-organization of such
vector-specific competition based on a ‘fire-together, wiretogether’ principle: As shown in Eqs. (15) and (18) below,
all plan cells coding for vector ðx; yÞ receive a common
excitatory input ksfp ½Sxy 2 0:25þ from SC zone ðx; yÞ and a
common inhibitory input Fxy
from FEF post-saccadic cells
excited by SC zone ðx; yÞ: For these reasons, the model
presented here includes between-zone competition between
cells coding for the same vector. However, different saccade
vector plans may simultaneously be highly active in
different zones, to provide alternative plans from which to
Activity Fxyi
of FEF plan layer cell xyi was defined by
d ðPÞ
F ¼ 500½ð1 2 Fxyi
ÞFxyi 2 ðFxyi
þ 0:4ÞFxyi
dt xyi
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
where Fxyi
is the total excitatory input to a plan layer cell,
and Fxyi is the total inhibitory input. The excitatory input
of (14) was defined by
¼ Fxyi
þ 0:025qðFiðGÞ ; 0:15Þ þ kfpr f ðPÞ ðFxyi
þ 0:1 j Wjxyi
Tj þ ksfp ½Sxy 2 0:25þ :
qðx; aÞ ¼
if x , a
if x $ a
ð½xþ Þ8
ð0:5Þ8 þ ð½xþ Þ8
f ðPÞ ðF11m
2 0:6þ
It allows background noise to be filtered out while
even small suprathreshold signals can generate robust
activity. Parameter kfpr of (15) scales the FEF plan layer
recurrent excitation, f ðPÞ ðFxyi
Þ; which is restricted to the
cell of origin. The sigmoid signal function f ðPÞ in (15) was
defined by
f ðPÞ ðxÞ ¼
of (14) were defined by:
The inhibitory inputs Fxyi
¼ 0:06 þ 5Fxy
þ kfpri ð f ðPÞ ðFxym
ÞÞ þ kfpi ðfjðWÞ ðFxyi
ÞÞ þ kfop ð
2 f ðPÞ ðFxyi
from the FEF input layer was defined in (12).
Input Fxyi
Term Fi is input from layer VI, defined by (13). The
threshold signal function qðx; aÞ that transforms FiðGÞ was
defined by
With Fxyi
as its argument, the steep sigmoid specified by
(17) ensures that recurrent excitation within the plan layer
can result in self-sustaining activity when a plan layer
cell’s activity exceeds 0.5 due to inputs from layer VI, ITa,
and/or SC.
Term Tj in (15) is input from ITa that is defined by
(5). It is multiplied by the adaptive synaptic weight Wjxyi
that is defined by (56). This weight is non-zero only for
projections to FEF What – Where (not Where –Where)
GCZs. This link allows a category representation in ITa
to learn to activate a saccadic plan. The term Sxy (defined
by (11)) is from the SC (Lynch, Hoover, & Strick,
1994). The rectified signal ½Sxy 2 0:25þ grows linearly
with Sxy above the signal threshold 0.25 but is zero for
all Sxy below 0.25. Parameter ksfp is the SC-to-FEF plan
layer synaptic strength.
Fixation-related plan cells are those cells with
Cartesian coordinates ðx; yÞ ¼ ð1; 1Þ: There were three
such cells, one in each FEF zone. For saccade-related
cells, kfpr ¼ 0:18; ksfp ¼ 8; for fixation-related cells,
kfpr ¼ 0:0; ksfp ¼ 0:0: Thus saccade-related, but not
fixation-related, cells can receive movement-related
corollary discharges from the SC and exhibit selfsustaining recurrent excitation.
Here Fxy
is inhibition from FEF postsaccadic cells,
defined below by (26). The parameter kfpri scales FEF plan
layer recurrent inhibition from same-direction saccaderelated cells in other zones. Recurrent inhibition prevents
multiple representations of the same saccade vector from
being active, to prevent interference between different
GCZs while strategies are learned. For plan cells in the
fixation and target feature zones, which can mediate
Where – Where transformations, kfpri ¼ 0:1: However,
kfpri ¼ 0:0 for cells in the What – Where zone, GCZo:
Since this zone lacks the positional inputs of the Where–
Where zones, interference during learning will not occur. Of
the final two terms in Eq. (18), only one is non-zero for any
single cell, because the model’s saccade- and fixationrelated plan cells have different patterns of inhibitory
afferents. The parameter kfpi scales recurrent inhibition from
saccade-related cells to saccade-related cells tuned to
different directions, indexed by the coordinates ðx; yÞ: The
parameter kfop scales the FEF output layer and plan layer
inhibition to fixation-related plan cells. Thus, for saccaderelated cells, kfpi ¼ 0:1; but kfop ¼ 0:0: For fixation-related
cells, kfpi ¼ 0:0; but kfop ¼ 1:0: The summation index k
denotes the zone ðk ¼ 1; 2; 3Þ of cells representing motor
error vectors ðp; qÞ:
In the fourth term of (18), the lateral inhibitory signal,
fjðWÞ ðFxyi
Þ; takes a different form for Where – Where
(indexed by j ¼ 1) and What –Where (indexed by j ¼ 2)
cells. Although Where– Where plans encoding the same
saccade vector inhibit each other across zones, Where–
Where plan cells do not inhibit What – Where plan cells.
Lateral inhibition from saccade-related plan cells to
saccade-related plan cells in either Where –Where zone,
was defined by
f1ðWÞ ðFxyi
f ðPÞ ðFpqk
k[V ðp;qÞ–ð1;1Þ
where V is the set of all plan layer cells in the Where–
Where zones, for which index k ¼ 0 and 1. (More
generally, the indices i and k refer to zones in the cortex
and BG, respectively. Use of separate indices
allows computation of the interaction among zones, e.g.
in Eq. (52)).
In (18), the subtraction of f ðPÞ ðFxyi
Þ from f1ðWÞ ðFxyi
fpr ðPÞ ðPÞ
excludes self-inhibition. Excitatory term k f ðFxyi Þ of
(15) and inhibitory term kfpi ðf1ðWÞ ðFxyi
Þ 2 f ðPÞ ðFxyi
ÞÞ of (18)
together define a recurrent on-center, off-surround network,
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
which acts to normalize and contrast-enhance network
activity (Grossberg, 1973).
For cells in the What –Where zone, kfpri ¼ 0:0 in (18),
and the lateral inhibitory signal f2ðWÞ ðFxyi
Þ was
f ðPÞ ðFpqi
f2ðWÞ ðFxyi
Lateral inhibition came only from saccade-related cells
of the same zone.
Model FEF output cells correspond to FEF movement
or presaccadic cells, and are predicted to reside in layer
Vb; see Figs. 2 and 6a. They are divided into fixationðOÞ
and saccade-related categories. Their activity Fxyi
defined by:
d ðOÞ
F ¼ 125½ð1 2 Fxyi
ÞFxyi Fi 2 ðFxyi
þ 0:6ÞFxyi
dt xyi
excites the output cells and Fxyi
where Fxyi
them. The arousal signal Fi from layer VI, defined by
(13), multiplies Fxyi
; which was defined by:
¼ 1:5½Vk 2 0:5
þ 0:4qðFxyi
;0:2Þ þ ksfo ½Sxy 2 0:4þ :
In (22), 1:5½Vk 2 0:5þ is the thalamocortical input;
see (38). It conveys a GO signal from the BG if thalamic
activity Vk exceeds 0.5. Subscript k of Vk has the same
range (1 – 3) as i of Fxyi
because there is a one-to-one
relationship between FEF zones and associated thalamic
sites controlled by distinct BG channels. Signal function
q(x,a) was defined in (16). Input Fxyi
is from the plan
layer and Sxy is from the SC. The product of Fxyi
Fi in (21) makes output contingent on a conjunction of
arousal with plan layer, thalamocortical, and SC inputs.
In particular, corollary discharges from the SC cannot
activate output layer cells whose corresponding zone is
not active. Parameter ksfo in (22) is the strength of the
SC’s projection to the FEF output layer. For saccaderelated cells, ksfo ¼ 10; for fixation-related cells, ksfo ¼ 0:
Therefore, a corollary discharge from the SC can excite
saccade-related but not fixation-related cells.
The inhibitory input Fxyi
in (21) differs for fixationrelated and saccade-related output cells. The latter are
inhibited by both postsaccadic cells and output layer cells
that are tuned to other saccadic vectors. The inhibitory
signals Fxyi
to saccade-related FEF output cells, for which
ðx; yÞ – ð1; 1Þ; were defined by:
¼ 0:3 þ 6Fxy
f ðOÞ ðFpqk
Þ 2 f ðOÞ ðFxyi
where Fxy
is input from postsaccadic cells defined by (26),
the summation index k denotes zone, and the output layer
signal function f ðOÞ was a sigmoid:
f ðOÞ ðxÞ ¼
ð½xþ Þ10
ð½xþ Þ10 þ 0:410
Because of the steep shape of the sigmoid in (24), output
layer cell activity must exceed 0.4 to inhibit other output
layer cells. The high threshold for competition among FEF
output cells creates an interval—before any cells exceed this
threshold on the basis of dual inputs from the plan layer and
the thalamus—during which FEF output cells can be
transiently activated by visually driven signals arriving via
the plan layer. Such a transient visual response is seen in the
presaccadic movement cell shown in Fig. 9D.
The inhibitory signal to fixation-related FEF output cells
was defined by
¼0:3 þ 6Fxy
f ðOÞ ðFpqk
Þ 2 f ðOÞ ðF11i
þ 10
2 0:6þ :
Again, summation index k denotes zone. The final
term in (25) allows saccade-related FEF output cells to
inhibit fixation-related output cells via inhibitory interneurons. Fixation-related plan and output cell activities,
specified in Eqs. (14) – (18) and (21,22, 25), respectively,
begin to shut off immediately prior to a saccade due to
the inhibition from saccade-related FEF plan and output
FEF postsaccadic cell activities Fxy
were defined by:
d ðXÞ
F ¼ 500ð1 2 Fxy
Þ½Sxy 2 0:6þ 2 10Fxy
dt xy
Postsaccadic cells delete executed plans; see Eq. (18).
They are excited by the corresponding SC cell activities
Sxy; see Eq. (11). The SC signal was modeled by
thresholding the SC activity. The 0.6 threshold for
exciting them is also the threshold of saccade initiation
in the SC.
6. Model BG and PNR-THAL cells
Cortical afferents to the striatum of the BG include
projections from inferotemporal cortex (Hoesen, Yeterian,
& Lavizzo-Mourey, 1981; Steele & Weller, 1993),
parietal cortex (Cavada & Goldman-Rakic, 1991), and
frontal cortex (Parthasarathy, Schall, & Graybiel, 1992;
Strick et al., 1995). All three classes of afferents
project to the BG model. BG components that ultimately
project to the SC are denoted by the symbol G, and
those that ultimately return to frontal cortex via the PNRTHAL are denoted by the symbol B. The model
coordinate systems are also distinct because the thalamus-projecting population codes a zone’s choice,
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
while the colliculus-projecting population codes motor
The SC-projecting striatal SPN direct pathway activity
(Figs. 2 and 6b) was defined by:
d ðSDÞ
¼ 30½ð1 2 GðSDÞ
2 ðGðSDÞ
xy ÞGxy
xy þ 0:58ÞGxy ;
dt xy
where the excitatory input is GðSDEÞ
; and the inhibitory input
xy : The excitatory signal was defined by:
¼ N þ 75½Pxy 2 0:25þ þ 100 f ðOÞ ðFxyi
where N refers to a dopamine burst (Ljungberg, Apicella, &
Schultz, 1992), which can activate the striatum and
modulate corticostriatal reinforcement learning (Wickens
et al., 1996). Signals from parietal cortex ½Pxy 2 0:25þ ; and
FEF output cells, f ðOÞ ðFxyi
Þ; promote opening of the BG-toSC gate to allow execution of the plans that they represent.
The model assumes that the excitatory inputs are strong but
specific: they are pooled only across parietal and FEF sites
that represent the same motor vector. In contrast, the
inhibitory inputs are convergent: they are pooled across
parietal and FEF sites regardless of the motor error vector
represented. The inhibitory input GðSDIÞ
was defined by:
Gxy ¼ 1 þ 20ð ½Ppq 2 0:25 þ
f ðFpqi ÞÞ;
which models fast-acting feedforward striatal inhibitory
interneurons (Koos & Tepper, 1999; Wilson et al., 1989)
that are excited by convergent parietal cortex and FEF
outputs from all positions ðp; qÞ and zones i: Because of the
relative strengths of the inputs in (28) and (29), a planrepresenting saccade vector ðx; yÞ can always activate the
striatum if the main source of inhibition in the summations
of (29) comes from cells at grid coordinate ðx; yÞ: This
condition will be met when the cortical competition resolves
in favor of one winning plan. Otherwise, inhibition can
overwhelm excitation; see Fig. 3. Also noteworthy is that
Eq. (29) does not include feedback inhibition from SPNs.
This is consistent with electrophysiological data of Jaeger
et al. (1994) indicating that such inhibition is weak. The
theory does not, however, preclude a role for feedback
inhibition; see Eq. (41).
Colliculus-controlling striatal SPN activity GðSIÞ
xy (Figs. 2
and 6b) was defined by:
d ðSIÞ
G ¼ 30½ð1 2 GðSIÞ
xy Þ10NSxy 2 ðGxy þ 0:58Þ;
dt xy
where N corresponds to a transient depression in the
dopamine signal (a dopamine dip), as described by
Ljungberg et al. (1992). The excitatory action of N is
attributable to reduced binding of inhibitory D2 receptors on
indirect path SPNs. Excitatory input, Sxy ; from the SC
informs the indirect channel that the SC region that is gated
by the corresponding direct channel is being activated.
As noted for the analogous case of thalamo-striatal feedback
in hypothesis 7, such an input to the indirect channel can
serve as a training signal for other excitatory afferents to
SPNs; see Eq. (34). However, no such afferents were needed
to SPNs represented by Eq. (30) in the present simulations.
The constant term (0.58) of Eq. (30) specifies tonic
(Figs. 2
Colliculus-controlling GPe activity GðGPeÞ
and 6b) was defined by:
d ðGPeÞ
¼ 30½0:5ð1 2 GðGPeÞ
dt xy
2 ðGðGPeÞ
þ 1Þð0:2 þ 0:8½GðSIÞ
xy Þ:
The first term of (31) represents a combination of
intrinsic excitation and tonic excitation from the STN. Input
from the striatal indirect pathway SPN provides
SNr activity GðSNrÞ
(Figs. 2 and 6b) was defined by:
d ðSNrÞ
¼ 100ð1 2 GðSNrÞ
þ 1Þð54½GðSDÞ
xy Þ 2 ðGxy
xy dt xy
þ 80½GðGPeÞ
þ Þ:
The first term of (32) represents a combination of
intrinsic excitation and tonic excitation from the STN. Both
the striatal direct pathway SPN ðGðSDÞ
xy Þ and the GPe ðGxy
provide inhibition.
The thalamus-projecting BG system differs from the SCprojecting system in having adaptive cortico-striatal pathways. The thalamus-projecting striatal SPN direct pathway
activity BðSDÞ
(Figs. 2 and 6b) was defined by:
d ðSDÞ
Bk ¼50½ð1 2 BðSDÞ
Þð70 ½Fxyi
2 0:33þ Wxyik
þ 8 Tj WjkðTSDÞ þ N þ 2½Vk þ Þ
þ 0:58Þð1 þ 1:17ð
2 ðBðSDÞ
Vi ÞÞ:
Subscript k ranges from 1 to 3 to denote the three
thalamus-controlling BG channels associated with the three
FEF zones. The FEF plan layer activities Fxyi
; defined in
(14), and the anterior IT activities Tj ; defined in (5), provide
learned excitation to striatal direct pathway SPNs. The
adaptive weight Wxyik
is defined by (53), and the adaptive
weight Wjk
is defined by (54). A dopamine burst N
provides a dopamine-mediated enhancement of striatal
activity via D1 receptors. Thalamic inputs Vk provide
excitatory feedback that can help train FEF and ITa inputs to
activate the striatal direct pathway. Activity in the FEF plan
layer and thalamus also use feedforward striatal inhibitory
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
interneurons (the last two terms in (33)) to inhibit striatal
direct pathway activity, as in (27) and (29).
Striatal indirect pathway SPN activity BðSIÞ
(Figs. 2
and 6b) was defined by:
d ðSIÞ
k þ Þ
Bk ¼30½ð12BðSIÞ
Wxyik ½Fxyi 20:33þ þ10N½V
k Þ70
k þ0:58Þð1þ0:17
ðPÞ þ
Learned excitatory inputs from the FEF plan layer Fxyi
are adaptively weighted by the Wxyik and add with
excitatory input Vk from the thalamus that is enhanced
when a dip ðN.0Þ
occurs in the dopaminergic input, which
acts on D2 receptors. Indirect pathway SPNs are also
suppressed by feedforward inhibitory interneurons, which
are driven by FEF plan layer inputs Fxyi
: The adaptive
weights Wxyik in (34) are defined by (52).
The SNr/GPi receives input from the STN, the striatal
direct pathway projections, and the GPe. The SNr/GPi
activity BðGPiÞ
(Figs. 2 and 6b) was defined by:
d ðGPiÞ
¼100½ð1 2 BðGPiÞ
Þð0:77 þ 2½BðSTNÞ þ Þ
dt k
2 ðBðGPiÞ
þ 1Þð0:54½BðSDÞ
þ þ 0:8½BðGPeÞ
þ Þ:
With ðp;qÞ–ð1;1Þ; the summation index spans inputs
from saccade-related, but not fixation-related, cells.
STN hyperactivity in Parkinson’s disease has, in the
past (Wichmann & DeLong, 1996), been attributed to GPe
hypoactivity and the subsequent loss of GPe-STN inhibition. This argument has supported the prevailing model
of BG function in which the GPe-STN projection is an
essential link in the indirect pathway. However, Levy et al.
(1997) reviewed evidence that GPe activity is not
significantly reduced in Parkinsonism, suggesting that
loss of GPe-STN inhibition may not be sufficient to
account for STN hyperactivity in Parkinson’s disease.
Moreover, dopaminergic lesions directly in the STN, which
contains D1 and D2 receptors (Brown et al., 1979; Martres,
Bouthenet, Sales, Sokoloff, & Schwartz, 1985), result in
five times the STN hyperactivity induced by GPe lesions
alone (Hassani et al., 1996), although the precise
mechanism for this is not clear. Therefore we have
modeled the GPe-STN projection as weak relative to the
GPe-GPi/SNr projection (Parent & Hazrati, 1995; Smith &
Bolam, 1990).
Activities Vk in the PNR-THAL, e.g. VA and VLo, are
excited by afferents from cortical layer VI (the category
layer, FkðGÞ ) and inhibited by the GPi ðBðGPiÞ
Þ: Thalamic
activity Vk (Figs. 2 and 6b) was defined by:
V ¼400½ð1 2 Vk Þð0:5½FkðGÞ 2 0:47þ þ f ðVÞ ðVk Þ þ vðtonicÞ Þ
dt k
2 ðVk þ 0:1Þ½0:2 þ 3:4½BðGPiÞ
2 0:2þ
By (35), the GPi receives tonic excitation and glutamatergic excitation BðSTNÞ from the STN, and is inhibited by
striatal direct pathway SPN outputs BðSDÞ
and GPe outputs
the SNr rather
than GPi projects to the PNR-THAL, which projects to FEF.
These SNr projections are distinct from SNr-to-SC projections modeled by GðSNrÞ
in Eq. (32) above (Kemel, Desban,
Gauchy, Glowinski, & Besson, 1988).
The GPe activity BðGPeÞ
(Figs. 2 and 6b) is excited by the
STN output BðSTNÞ and inhibited by the striatal indirect path
SPN outputs BðSIÞ
d ðGPeÞ
¼ 30½ð1 2 BðGPeÞ
Þð0:46 þ 0:25BðSTNÞ Þ
dt k
2 ðBðGPeÞ
þ 1Þð0:2 þ 0:8½BðSIÞ
k Þ:
The STN activity B (STN ) (Figs. 2 and 6b) is excited by
afferents from layer Vb (the output layer, F ðOÞ ) of the FEF
and inhibited by the GPe outputs BðGPeÞ
¼25½ð12BðSTNÞ Þð0:016þ10
20:5þ Þ
i ðp;qÞ–ð1;1Þ
þ :
f ðVÞ ðVi ÞViðXÞ Þ;
where the feedback signal function f ðVÞ was defined by
f ðVÞ ¼ ð½xþ Þ2 :
The term vðtonicÞ is 0.0 if the thalamic cell projects to a
Where– Where FEF zone (i.e. k , 2) and 0.1 if the thalamic
cell projects to a What –Where FEF zone (i.e. k ¼ 2). This
small tonic level biases the system to use What – Where FEF
plans initially during learning. For example, in learning the
delay task, the model first attempts a saccade to the target
that was learned in the gap task. This target is recalled via a
learned What –Where transformation. Subsequent punishment trains the model to suppress the response learned in the
gap task.
Thalamic recurrent inhibition, which is well established
in the visual thalamus (Lo & Sherman, 1994), mediates a
local winner-take-all competition that allows only the
thalamocortical projection of one FEF zone at a time to
generate output. Since lateral inhibition within a zone in
the plan layer allows only one plan to be active per zone,
this helps ensure that multiple conflicting plans are not
simultaneously executed. In the thalamus, newly formed
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
plans can shut off a prior plan and instate themselves as
current winner if the ability of the prior winner to
maintain itself wanes through time. This will occur if selfexcitation and/or recurrent surround inhibition use habituative signals. The current implementation uses only the
latter. Thus, a transmitter level, symbolized VkðXÞ in (38)
and defined in (40), is assumed to mediate GABA-ergic
lateral inhibition and to habituate (inactivate, or depress)
with continued firing. As a result, a new plan can be
instated and can effectively inhibit a previously active
plan whose inhibitory signals have habituated. Thalamic
habituative transmitter VkðXÞ was defined (cf., Grossberg,
1982) by:
d ðXÞ
V ¼ 0:25ð1 2 VkðXÞ Þ 2 12:5VkðXÞ ½Vk 2 0:4þ ;
dt k
where VkðXÞ habituates via term VkðXÞ ½Vk 2 0:4þ if activity
Vk . 0:4; but accumulates to a baseline of 1.0 via term
ð1 2 VkðXÞ Þ when the thalamic signal Vk is inactive. In
vivo, this habituative dynamic may be attributable to
short-term depression of excitatory synapses on inhibitory
interneurons of the thalamus.
7. Second messenger traces and working memory
Reward signals subserving reinforcement learning arise
after an action has been generated, with a delay of
hundreds of milliseconds or even seconds. However,
movement-related cell discharges shut off rapidly after a
movement is initiated (Figs. 9C and F). Thus, reinforcement learning signals must modify synapses for which the
pre- and post-synaptic cells were previously, but are no
longer, active. In order for credit or blame to be properly
assigned to synapses on non-discharging cells that
previously helped activate a movement, a record or trace
of recent cell activity persists after spiking ceases
(Dominey et al., 1995; Fiala, Grossberg, & Bullock,
1996; Grossberg & Merrill, 1992; Grossberg & Schmajuk,
1987; Houk, Adams, & Barto, 1995; Klopf, 1982; Sutton &
Barto, 1981). Such traces can take the form of intracellular
second messengers, which are critical for some aspects of
learning in many plastic brain areas (Hemart, Daniel,
Jaillard, & Crepel, 1995; Otani, Auclair, Desce, Roisin, &
Crepel, 1999; Sung, Choi, & Lovinger, 2001; Vezina &
Kim, 1999). Other options, such as working memory
representations of previous movements, occur in situations
where the cell activation can itself be sustained over the
interstimulus interval.
Trace variables that embody these proposed second
messenger effects were introduced in the BG as follows. To
support reinforcement learning for awhile after direct
pathway activity ceases, a trace, or transient record, of
recent striatal direct pathway activity was carried by second
messenger activity B ðSDÞ
(Fig. 6b):
d ðSDÞ
¼750ð1 2 B ðSDÞ
2 0:4þ
dt k
2 B ðSDÞ
ð0:75 þ 75 f ðBÞ ðBðSDÞ
The threshold 0.4 ensures that the trace forms only if
is sufficiently large. The trace B ðSDÞ
influences learning
of the weights Wxyik
in (33); see (53). The final term in (41)
formalizes the assumption that GABA-ergic recurrent
collaterals of striatal direct pathway projection neurons
suppress second messenger-based records of activity within
neighboring striatal projection neurons. This hypothesis
may explain why collaterals of striatal SPNs exist despite
their apparent failure to produce potent lateral inhibition
(Jaeger et al., 1994). The inhibitory feedback signal function
f ðBÞ in (41) was defined by:
f ðBÞ ¼ ð½xþ Þ2 :
Learning by the adaptive weights
in (34) that
impinge on indirect pathway cells suppress a negatively
reinforced response. Second messenger activity B ðSILÞ
as a trace of thalamic input Vk to striatal indirect channel
SPNs, and determines whether the weights Wxyik
from the
FEF plan layer to these SPNs are eligible to learn, should
negative reinforcement occur; see (52):
¼750ð1 2 B ðSILÞ
Þ½Vk 2 0:5þ
dt k
ð0:75 þ 75 f ðVÞ ðVi ÞViðXÞ Þ:
Here Vk is thalamic activity defined by (38), and ViðXÞ is
transmitter that habituates with continued firing, as defined
in (40). The signal function f ðVÞ was defined by
f ðVÞ ¼ ð½xþ Þ2 :
The second term in (43) models the hypothesis that the
trace is suppressed in neighboring cells via habituative
feedforward inhibition from striatal interneurons (Kawaguchi, 1997; Kawaguchi, Wilson, Augood, & Emson,
1995) that are excited by thalamic signals Vk : Simulations
showed that this striatal habituation to Vk could be
identical to the thalamic habituation to Vk ; which was
already modeled in (38) and (40). For convenience, the
same variable, ViðXÞ ; was therefore used in both (38)
and (43).
A trace of FEF plan layer activity F ðPÞ
xyi (Fig. 6a) was
specified by:
d ðPÞ
F ¼ 15ð1 2 F ðPÞ
xyi Þ½Fxyi 2 G
dt xyi
2 0:75F ðPÞ
xyi :
pt þ
Eq. (45) compactly describes the dynamics of two
physically distinct traces of FEF plan cell activity. One of
these traces modulates learning in anterior IT projections
to the plan layer (see Eq. (56)). It is naturally interpreted
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
as a process localized in the ITa-recipient dendrites of
FEF cortico-striatal cells. The second trace (whose
dynamics are also described by (45)) modulates learning
(Berretta et al., 1997) in the plan layer projection to the
striatal direct pathway (see Eq. (53)). It is naturally
interpreted as a process in the axon terminals of
projections from FEF plan layer cells to striatal direct
pathway SPNs. For Where –Where cells, G pt ¼ 0:4; for
What – Where cells, G pt ¼ 0:15 to compensate for
weaker What – Where plan cell activity by reducing the
threshold for trace activation.
A plan layer cell is prevented from learning to activate
those striatal indirect-pathway SPNs that can prevent the
same plan from being executed. Otherwise, when a
dopamine dip occurred, a plan could learn to prevent
itself from ever being executed (a kind of learned
helplessness). However, plans can block each other’s
execution via learned FEF-to-indirect pathway links. The
model accomplishes these goals using traces within plan
layer cells that gate plan-layer to indirect-pathway
learning. These traces create eligibility only for recently
active but unexecuted plans because plan execution
generates feedback, via model GABA-ergic FEF postsaccadic cells, that resets activities and traces only within
plan and output cells responsible for execution of the
current response. This trace of plan layer activity, F ðPAÞ
xyi ;
(Fig. 6a) was defined by:
d ðPAÞ
pt þ
F ¼15ð12 F ðPAÞ
xyi Þ½Fxyi 2 G 2 Fxyi ½0:75þ75Fxy :
dt xyi
Like the trace in (45), this trace is activated only if FEF
plan layer activity Fxyi
; which is defined in (14), satisfies
Fxyi . G pt : However, unlike other trace variables in the
model, which decay slowly once their source cell’s activity
falls below G pt ; this trace collapses quickly when its source
cell is shut off, notably when an FEF post-saccadic cell
generates an inhibitory signal Fxy
; defined in (26). For
convenience, the dependence on this inhibitory event is
shown directly in the final term of Eq. (46), where a
function that became large when Fxyi
nears zero would also
work. This captures the hypothesis that a small activity in
the source cell is necessary to prevent rapid collapse of this
type of trace. This trace is interpreted as a process in
cortico-striatal axon terminals that abut dendrites of
indirect pathway SPNs. Consistent with this proposal,
Berretta et al. (1997) showed that applying a GABA
agonist in motor cortex prevents learning-related gene
expression in projections to the striatal indirect, but not
direct, pathways. The model captures this functional
difference by introducing the dependence on cortical
inhibition in (46) but not in (45).
A trace of layer VI activity F ðGÞ
(Fig. 6a) was used to
support reinforcement learning, after movement completion, of adaptive weights between PFC context representations and FEF zones that can be primed by PFC inputs to
layer VI. This trace variable was defined by:
d ðGÞ
2 0:5þ
F ¼750ð1 2 F ðGÞ
i Þ½Fi
dt i
2 F ðGÞ
f ðFk ÞÞ:
i ð0:75 þ 75
The inhibitory summation term in (47) represents lateral
inhibition among layer VI cells. The inhibitory signal
function f ðGÞ in (47) was defined by
f ðGÞ ¼ ð½xþ Þ4 :
Eqs. (47) and (48) ensure that a synapse between a PFC
fiber and a layer VI cell will remain eligible to learn until the
trace either passively decays or is actively shut off by
competitive activity in a neighboring layer VI cell.
A trace of ITa activity T j (Fig. 6a) was defined by
d T ¼ 750ð1 2 T j Þ½Tj 2 0:4þ 2 0:75T j
dt j
and may be realized by second messenger activity within
terminals of anterior IT cell synapses with FEF plan cells.
Although the What pathway trace in (49) facilitates
learning, it does not control performance. Working memory
areas in the PFC provide a basis for persistent activation of
contextual representations that is useful for controlling
performance, specifically which premotor zones are set in
an active or primed state. As noted in (47), model PFC What
representations learn to activate layer VI cells via longrange cortico-cortical projections. PFC What working
memory activities Cj (Figs. 2 and 6a) were defined by:
C ¼ 30½ð1 2 Cj Þð1:5I ðMÞ þ Tj þ 4f ðCÞ ðCj ÞÞ 2 ðCj þ 0:3ÞÞ1
dt j
þ 0:35 f ðCÞ ðCk ÞÞÞ:
Motivational input I ðMÞ is either 0 or 1. Model PFC
representations require such an input I ðMÞ ; assumed to be
from limbic areas, in order to become aroused. Then a
specific ITa signal Tj can activate a working memory
representation as follows. Positive feedback 4f ðCÞ ðCj Þ via
the sigmoidal signal function f ðCÞ ;
f ðCÞ ¼
x8 þ 0:68
is balanced by recurrent inhibition 0:35 k–j f ðCÞ ðCk Þ from
other cells. By (51), Cj activity must exceed approximately 0.6 for recurrent excitation to engage and
maintain Cj in a self-sustaining regime. Input Tj from
ITa cannot activate working memory alone, because it
cannot drive activity Cj above 0.6. However, when
motivational input I ðMÞ is active, the sum of the nonspecific I ðMÞ and the specific input Tj can drive one or
multiple prefrontal cells Cj above 0.6 and into the selfsustaining regime; cf. Grossberg (1971, 1982b). Once
self-sustaining, Cj persists only as long as at least one of
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
I ðMÞ and Tj is active. In a more complete model,
sustained activity in PFC might also depend on
reciprocal excitation between frontal and parietal cortices
(Chafee & Goldman-Rakic, 2000).
The anterior IT-to-striatal direct pathway weights
WjkðTSDÞ (Fig. 6b) in (33) were defined by:
¼ðN½T j 2 WjkðTSDÞ þ 2 0:1ðN þ NÞW
dt jk
qðB ðSDÞ
; 0:5Þ:
8. Adaptive weight equations
Five similar learning equations describe the functional
dependence of adaptive weight changes on the intracellular
traces of recent activity defined above. Learning of the planðPSIÞ
to-indirect pathway weights Wxyik
(Fig. 6b) in (34) was
defined by
¼ N½500qð
; 0:35Þ½qðF ðPAÞ
xyi ; 0:5Þ
dt xyik
2 Wxyik
2 Wxyik
generated by the
A negative reinforcement signal N;
omission of primary reward at the expected time during
training (Brown et al., 1999), allows recently active but
unexecuted plans, which are marked by plan layer traces
xyi of (46), to learn to suppress the previously selected
zone with synaptic efficacy Wxyik
: The zone is
suppressed via activity in striatal indirect pathway
SPNs, whose signals B ðSILÞ
; defined in (43), are still on
when signal N arrives. The signal function q of Eq. (16)
in learning rules mimics the voltage-dependent effect of
NMDA channel blocking with magnesium (Mayer,
Westbrook, & Guthrie, 1984). Noise below the threshold
is filtered out, while even a small activity above the
threshold causes a robust learning response.
Learning of the plan-to-direct pathway weights Wxyik
(Fig. 6b) in (33) was defined by:
¼ðN½F ðPÞ
xyi 2 Wxyik dt xyik
2 0:1ðN þ NÞW
; 0:5Þ:
Expansion yields two products on the right hand side.
The first product specifies a process of LTP: weights
increase toward F ðPÞ when a dopamine burst N coincides
with recent presynaptic activity F ðPÞ from the plan layer
and recent postsynaptic activity B ðSDÞ
: The second
product specifies a process of LTD: weights decrease
slowly when a dopamine burst or dip (N or N;
respectively) coincides with recent postsynaptic
activity B ðSDÞ
: that occurs in the absence of recent
presynaptic activity. Thus localized FEF planning
activities become potent exciters of the striatal direct
pathway if they predict striatal activations leading
to forthcoming rewards, but they can lose this
potency if they predict striatal activations leading
to punishment, or if reward is forthcoming in their
Here positive reinforcement N allows the traces T j of ITa
representations in (49), in conjunction with striatal direct
pathway SPN trace activity B ðSDÞ
; to drive learning in the
adaptive weights WjkðTSDÞ : Both positive ðNÞ and negative ðNÞ
reinforcement allow weight decay in response to irrelevant
stimuli, whose ITa representations are inactive when the
direct pathway and reinforcement are active. By selectively
augmenting only recently active and positively reinforced
inputs, the learning rules in Eqs. (53) and (54) prevent
irrelevant stimuli and plans from learning to open the BG
The PFC-to-FEF layer VI weights WkiðCGÞ (Fig. 6a) in (13)
were governed by
d ðCGÞ
jiðCGÞ Þ
¼ð500N½qðCj ;0:5Þ 2 WjiðCGÞ þ 2 0:1ðN þ NÞW
dt ji
£ qðF ðGÞ
i ;0:5Þ:
By (13), this weight can enable the PFC working memory
to learn to activate the appropriate zone in layer VI, and by
(55) the weight grows when the working memory activity Cj
and layer VI trace F ðGÞ
i are both active while a reinforcement
learning signal N is active. The weight declines when F ðGÞ
i is
active without Cj ; or if Cj predicts punishment.
The anterior IT-to-FEF plan layer (What – Where)
weights Wjxyi
(Fig. 6a) in (15) were defined by:
d ðTPÞ
ðTPÞ þ
¼ð500N½F ðPÞ
xyi 2 Wjxyi dt jxyi
2 0:1ðN þ NÞW
ÞqðT j ; 0:5Þ:
Here active ITa traces T j ; and active plan layer traces F ðPÞ
together drive learning of adaptive weights Wjxyi
reinforcement N occurs. The ITa projection reaches only
What-to-Where zone cells in the FEF plan layer, in keeping
with the topographical distribution of ventral and dorsal
stream afferents to FEF (Schall et al., 1995a). In other
words, Wjxyi
¼ 0 if i corresponds to a Where-to-Where plan
layer cell. Eqs. (55) and (56) explicate hypothesis 12, that
DA-ergic signals influence cortical reinforcement learning
and plasticity in a manner similar to the way they influence
reinforcement learning in the BG.
Simulations. Details regarding simulated inputs, of visual
stimuli and reward signals, are provided in the Results
section. The simulations were implemented with a Cþ þ
program that used the fourth order Runge-Kutta method for
integrating the system of differential equations.
Parameters. In many cases, parameters were found for a
given equation by specifying several equilibrium values of
the governed variable and specific corresponding inputs to
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
the governing equation. For example, the GPi is required to
have low activity (0.2) when the striatal direct pathway
afferents are active and the GPe is tonically active. This
provides one constraint. Second, it is required to have higher
activity (0.5) when the striatal direct pathway projection is
inactive and the GPe is tonically active. This provides a
second constraint. Finally it must have high activity (0.8)
when the striatal direct pathway is inactive and the indirect
pathway is active, resulting in a loss of GPe – GPi inhibition.
This provides a third constraint. Together, these three
constraints can be used to simultaneously fit as many
parameters. The constrained inputs here refer to all variables
that are not parameters, such as afferent signals from other
parts of the model. For a given governing equation, each
desired equilibrium value and input condition pair constitute
a constraint on the parameters. This equilibrium-fitting
approach required much less computational power than
fitting spike histograms to the model output curves. The
governing differential equation can be set to zero (equilibrium), and each constraint then yields an algebraic
equation in the unknown parameters. In principle, if the
number of constraints equals the number of unknown
parameters in a given equation, the resulting system of
equations can be solved exactly.
In practice, two gradient descent methods were used
to solve the equations. In the first method, the objective
function is the sum of the squared time derivatives
(which we refer to as STID) of the governing equation
for each constraint. The objective function is minimized
if the time derivatives are zero, indicating that equilibrium is achieved for all constraints. In the second
method, the objective function is the sum of the squared
equilibrium errors (which we refer to as SEE) between
the desired equilibrium value and the actual equilibrium
value for the constraint, given the parameter choice. The
two methods are not equivalent, because the time
derivative and equilibrium error are not linearly related,
due to the nonlinearity of the equations themselves. For
example, doubling the equilibrium error does not
necessarily double the time derivative. Although the
SEE method was more accurate in some cases, it was the
slower of the two because it required solving each
constraint for its equilibrium value at each iteration,
given the parameters. The STID method simply calculated the time derivative directly from the governing
equation. For this reason, STID was run to get an initial
parameter estimate, and then SEE was used to refine the
estimate. In cases where the constraints were not
especially complex, parameters were adjusted manually.
The equilibrium-fitting technique could not specify the
overall time constant for a given governing equation;
these were also adjusted manually. In cases where each
term in the governing equation contained a parameter to
be fitted, the solver tended to find the trivial solution,
corresponding to a zero constant multiplier for each term
in the equation. This was prevented by constraining
the sum of parameters to be normalized to a constant
value n equal to the number of parameters to be fit. For
all model variables, the parameters found were not
particularly sensitive, and they could be rounded up to
only a few significant digits. For example, the plan-tooutput layer weight scale factor in Eq. (22) was rounded
from 0.377 to 0.4, and the anterior IT to striatal scale
factor in Eq. (33) was rounded from 7.858 to 8. Even
when all of the model parameters were rounded to one or
two significant digits simultaneously, there was no
significant difference in model learning or performance
after vs. before parameter rounding, indicating a relative
insensitivity to parameter choice.
9. Results
Model simulations of oculomotor tasks. Learning and
performance of seven oculomotor tasks were simulated: the
five tasks shown in Fig. 1, and two additional ones described
below. With the exception of the fixation task, all the tasks
required generation of a saccade, and Fig. 7 summarizes the
activation dynamics implied by hypotheses 1 – 14 (and the
system of equations) for a typical plan-execute episode.
However, the cortical paths via which the BG gate was
opened, the need to use the indirect BG pathway to STOP
unrewarding gate opening, and the time required to open the
BG gate following fixation point offset, varied across tasks.
The TELOS model was trained on the five tasks that
required learning in a single order: saccade, fixation,
overlap, gap, and delay. (The tasks could be learned in
other orders, but no others were simulated.) The model was
required to correctly perform one task trial before proceeding to learn a subsequent task. Trials were given at 5 s
intervals, which provided sufficient time for the model
activity to subside and equilibrate between trials. Time t was
reinitialized to be zero at the beginning of each trial. At time
t ¼ 0; a motivational signal, I ðMÞ was always set to 1.0 and
input to the model dorsolateral PFC (the ‘What’ working
memory activity Cj in Fig. 2; also see Fig. 6a), which in turn
excited model FEF layer VI cells. These cells excited
overlying model FEF plan cells (layers II/III/Va), thereby
enabling subsequent inputs from IT and PPC to induce
reverberating activations in these plan cells. Input I ðMÞ
remained on until reward or punishment signals (N or N;
respectively) terminated at the end of the trial. Input to the
model of a visible external stimulus, Ixyj
; always set to 1.0
the internal input, Ij ; which excited the ITa (TE) stage
activity Tj, shown on the upper left of Fig. 2. (The
superscript asterisk marks the associated variable as
external to the brain.) Subscript ðx; yÞ indexes the input’s
visual field (grid) location, j ¼ 1 or 2 indexes its distinctive
feature value (to distinguish Fixation points from other,
Target, inputs), and its ITa-coded (feature-based and
position independent) classification. When a saccade was
performed, visual inputs shifted position 30 ms after
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Fig. 7. Model of a canonical plan-execute episode. (A) Top-down arousal from higher or more anterior cortical areas excites layer VI, which in
turn primes action plan representations in layer III/Va. (Layer VI-to-III projections are pre-set in the present model but could be learned.) The
combination of priming and sensory input supraliminally activates layer III/Va plan cells. (B) Layer III/Va activity persists after the offset of
sensory input and excites specific movement-related cells in layer Vb, though not sufficiently to supraliminally activate layer Vb cells. (C)
Layer III/Va plan cells excite the direct pathway of the BG, which removes the tonic inhibition of the thalamus. This disinhibition allows the
existing layer VI signals to thalamus to supraliminally activate disinhibited thalamic cells. (D) These thalamic cells excite layer Vb cells. The
combination of layer III/Va and thalamic inputs drives localized layer Vb activity above the threshold needed to generate output to the SC and
STN. (E) Buildup of activity in direction-tuned cells of the SC sends feedback to FEF that initially excites plan cells tuned to the same
direction. Once the buildup becomes large enough to initiate movement, the stronger feedback to FEF transiently activates postsaccadic cells,
which are modeled as inhibitory interneurons that shut off the movement-related activity in layers III/Va and Vb. This prevents perseveration of
already executed plans.
saccade initiation to bring the saccade’s goal into the grid’s
‘fovea’ at locus ðx; yÞ ¼ ð1; 1Þ: Featural information from
the shifted scene began updating the FEF representations
after a 50 ms. consistent with the empirical delay (see
Fig. 9).
The model output was determined by the state of the SC,
which was modeled (Fig. 5) as a map of ‘motor errors’
(Grossberg & Kuperstein, 1986; Lee et al., 1988), i.e. of eye
movement vectors which, if performed, would zero the
difference, or error between current and desired eye position
ðx; yÞ: The SC map was excited by the model’s PPC stage,
FEF output layer, or both (Figs. 2 and 6b). Whenever an SC
cell’s activity rose above 0.25, it began to excite FEF plan
cells tuned to the same direction. If the SC cell’s activity
reached a threshold of 0.60, it was deemed to have initiated a
saccade (Hanes & Schall, 1996) corresponding to the cell’s
motor error vector representation. Such initiation resulted in
excitation of FEF inhibitory interneurons capable of
suppressing FEF plan cell firing (Burman & Bruce, 1997).
After a 100 ms delay following saccade initiation, a reward
signal ðN ¼ 1:0Þ was given if the saccade was correct, and a
punishment signal ðN ¼ 1:0Þ was given if the response was
incorrect. The reinforcement learning signals N and N could
not be simultaneously active. They correspond to dopamine
bursts and dips, respectively, and they lasted 100 ms
(Ljungberg et al., 1992; Brown et al., 1999).
Application of the model to the saccade task (Fig. 1) is
summarized by the canonical plan-execute sequence of
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Fig. 7, in which there is no STOP signal generation in the
indirect pathway. At time t ¼ 0; the fixation light was
presented in the fovea: Ixyj
was set to 1.0 for ðx; y; jÞ ¼
ð1; 1; 1Þ; i.e. location (1,1) where j ¼ 1 indicates that the
input possessed the distinctive feature of a Fixation point.
The input IjðITÞ to ITa was set to 1.0 for j ¼ 1; to activate a
spatially-invariant, feature-based representation of the
fixation light. At t ¼ 200 ms; the fixation light shut off,
and the target Ixyj
appeared to the right, at ðx; yÞ ¼ ð2; 1Þ;
with j ¼ 2 to signify that the input possessed the distinctive
feature of a Target. This target input activates ITp and then
the corresponding target feature GCZ in FEF. Via the Target
cortical zone GCZt, the model initiated a saccade at t ¼
279 ms; so the RT between fixation light offset and saccade
onset was just 79 ms (Fig. 9). At t ¼ 309 ms; which is 30 ms
after saccade onset, the target moved to the fovea as a result
of the saccade, meaning that Ixyj
shifted from ðx; yÞ ¼ ð2; 1Þ
to ðx; yÞ ¼ ð1; 1Þ; where j ¼ 2. At t ¼ 379 ms, which was
100 ms after saccade initiation, reward was given by setting
N to 1.0 for 100 ms. At t ¼ 479 ms, the trial ended and all
inputs were shut off, including I ðMÞ ; Ixyj
; IjðITÞ ; and N. Thus,
on the first trial, the GCZt mediated correct performance of
the saccade task, which involves visually-guided saccades
with no STOP requirements. In addition, SC activation
provided a teaching signal that caused like-tuned planning
cells in the object cortical zone GCZo to be briefly excited.
Because of the predictive relationship between fixation light
onset, plan cell excitation in the GCZo, and generation of a
positive reinforcement signal 100 ms after performance of a
rightward saccade, an ITa-FEF link began to be learned
between fixation light identity and the plan for a rightward
saccade in the What – Where module—even though that
module had not generated the reinforced saccade. (This link
would have enabled anticipatory excitation of an FEF plan
for a rightward saccade in the saccade task if training of the
saccade task had continued; see below).
The fixation task (Fig. 1) was presented next. At t ¼ 0;
the fixation light appeared in the fovea. It activated the link
learned (as a consequence of the last rewarded trial in the
saccade task) from the ITa representation of the fixation
light to GCZo plan cell for a rightward saccade. The GCZo
then generated an anticipatory saccade, even before the
target light could excite a saccade through GCZt. This
erroneous response led to a punishment signal, N;
triggered a weight increment on the pathway from the FEF
fixation plan cell to the BG indirect pathway associated with
GCZo; see Fig. 2. The punishment signal also caused a
decrement in the ITa to FEF link that had caused the
anticipatory saccade. Consequently, on the next trial, the
anticipatory activation of the GCZo plan cell by fixation
light onset was no longer able to open the BG gate. At
t ¼ 200 ms, the distracting target appeared to the right at
ðx; yÞ ¼ ð2; 1Þ while the fixation light remained visible.
Distracter onset excited the rightward saccade plan cell in
GCZt, which was able to open its BG gate. The model thus
erroneously initiated a saccade to the distracter target, at
361 ms, which led to activation of the punishment signal.
This caused a large increment in the weight on the link from
the FEF fixation plan cell to the BG indirect pathway
associated with the GCZt. On the next trial, neither GCZt
nor GCZo were able to open their BG gates in response to
the fixation light. Consequently, the model was able to avoid
looking at the distracter, and to maintain fixation, on this
trial. The model was rewarded beginning at t ¼ 500 ms.
Because the fixation light was still on and activating the
learned pathway to GCZo, this reward further strengthened
the link (from 0.3 to 0.9) between the fixation stimulus
representation in ITa and the rightward saccade plan in
GCZo. However, the rightward saccade plan is not executed
while the fixation cue is present, because the fixation plan
has also learned to activate the indirect pathway, which
STOPs saccades driven by GCZo as long as the fixation cue
is present.
The overlap task (Fig. 1) was presented next and the
model performed correctly on the first trial. It did not
saccade immediately upon target onset, because the model
had previously learned to withhold a planned saccade
despite distracters while the fixation light remained on. It
did saccade at fixation light offset because then the fixation
plan vanished as a source of a STOP signal. The fixation
light appeared in the fovea at t ¼ 0: At t ¼ 200 ms, the
target appeared to the right at ðx; yÞ ¼ ð2; 1Þ: Fixation light
offset at t ¼ 500 ms removed the source of excitation of the
indirect pathway, and thereby released, at t ¼ 559 ms, a
correct saccade to the target. Because of the weights
acquired during the two prior tasks, both the GCZt and
GCZo helped generate this saccade. Both weights were
helpful because the response required in the overlap task
was the same as that required in both prior tasks. Following
the correct saccade, a reward was given, but only modest
further change occurred in model adaptive weights.
The gap task (Fig. 1) was presented with a 500 ms delay
between fixation light offset and target onset. At fixation
light offset, an anticipatory saccade was performed correctly
on the first trial. This was because the fixation light feature
activated a saccade plan of the appropriate direction,
mediated by the ITa-FEF pathway, which had been learned
in the saccade task and not unlearned in the intervening
tasks. Indeed, it had been strengthened in the fixation task.
The anticipatory saccade was generated only once the
fixation light shut off, because then the fixation light was no
longer able to drive the indirect pathway to suppress the
anticipatory saccade. The fixation light appeared in the fovea
at t ¼ 0 and shut off at t ¼ 200 ms. A saccade to the
anticipated target location at ðx; yÞ ¼ ð2; 1Þ was generated
via GCZo at t ¼ 340 ms. Reward was given and the trial
ended prior to t ¼ 700 ms (at which point the target would
have appeared had the anticipatory saccade not been
generated). Had the anticipatory saccade not occurred,
the appearance of the target would have driven
a visually-guided saccade to foveate it. This version of
the gap task differs from that often used, e.g. in studies of
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
express saccades (Fischer & Weber, 1993). In that version,
reward is withheld if the saccade is anticipatory. Although
this is intended to force the animal to wait until the target
stimulus appears, animals and humans nevertheless generate some anticipatory saccades, and our simulations
indicate that the model is also capable of learning to
generate anticipatory saccades.
The delay (or memory) task (Fig. 1) was next presented.
The fixation light was visible in the fovea from t ¼ 0 until
t ¼ 800 ms. The target appeared in the upper right at
ðx; yÞ ¼ ð2; 2Þ and was visible from t ¼ 200 until
t ¼ 500 ms. When the fixation light shut off, two saccade
plans had formed in the plan layer: one was maintained by
recurrent excitation in GCZt to represent the remembered
target location; the other, in GCZo, represented the fixation
light activated, ITa-FEF mediated, anticipated location of
the target that had been used in prior tasks. At t ¼ 987 ms,
an ‘anticipatory’ saccade to the previously rewarded locus
was erroneously initiated by GCZo, and punishment was
given. This caused a large weight increment in the STOP
pathway between the GCZt plan cell and the indirect
pathway controlling GCZo. On the next trial, this new link
stopped the previously learned tendency to initiate an
anticipatory saccade via GCZo, and thus a saccade was
correctly initiated after the delay by GCZt. The model’s
saccade to the remembered target location at t ¼ 969 ms
was rewarded. The brief 300 ms delay between target offset
and fixation light offset was chosen for convenience, and
could have been much longer for a wide set of parameters.
Because the model is noiseless, there is no limit on how long
it can maintain a plan cell activation representing the target
locus, provided that recurrent excitation is strong enough
and that no other salient visual inputs are presented during
the memory interval.
Since the model had now learned to perform each of the
tasks correctly, the training ended, and the next trials
marked a final testing phase of the simulation. The state of
the simulation was saved at the beginning of the next trial
and served as the initial condition for testing each of the
tasks. The tasks were presented as during training and were
all performed correctly. RTs (measured from fixation
light offset to saccade onset) from this final test are listed
in Table 3. They are similar to those measured on the first
correct trial during initial training of each task.
Subsequent to training, a visual discrimination task was
performed to assess choice between simultaneously presented peripheral stimuli, one possessing the fixation light
feature, and one the target light feature. The usual fixation
light was first presented in the fovea at t ¼ 0: At t ¼ 200 ms,
Table 3
Simulated saccadic RTs for tasks
Reaction Time (ms)
a stimulus that also possessed the fixation light feature
appeared in the upper left at ðx; yÞ ¼ ð0; 2Þ; and a stimulus
possessing the target feature simultaneously appeared in the
upper right at ðx; yÞ ¼ ð2; 2Þ: GCZf activity quickly began to
reflect both the foveal and the peripheral fixation feature
input. Competition between these two GCZf activities
resolved in favor of the peripheral activity. Since GCZf
received a permissive signal from the PNR-THAL, at
t ¼ 402 ms, with the foveal fixation light still on, the model
initiated a saccade to the upper left; i.e. to the peripheral
stimulus that possessed the fixation light feature. This
response was ‘correct’ given past training, because the
model had never been punished for saccading to such a
stimulus, but had been punished for saccading to stimuli
possessing the target feature whenever the fixation light
remained on. At t ¼ 432 ms, the stimulus at the upper left
moved to the fovea, and all other stimuli were extinguished.
The RT appears under ‘choice’ in Table 3.
Also, subsequent to training, the model was tested on a
countermanding saccade task (Hanes, Patterson, & Schall,
1998; Logan & Cowan, 1984). This was essentially a
saccade task, except that on a fraction of trials, the fixation
light reappeared after a brief time off. The reappearance
constituted an imperative STOP signal. It instructed the
system to cancel the planned saccade and instead maintain
fixation. Provided that the STOP signal was not presented
too late, the model was able to cancel the saccade by using a
fixation light activated STOP signal mediated by the BG
indirect pathway. The last two tests illustrate that the model
can perform some tasks on which it was not trained.
Adaptive weights were adjusted as described above, in
accord with learning equations detailed in the Mathematical
Model section. An algorithm (see Mathematical Model) had
previously adjusted other model parameters to minimize
the difference between model cell dynamics and
the observed cell dynamics of a large number of
electrophysiological cell types identified in published
experimental studies of the modeled areas. Table 4 lists
electrophysiological reports that were used to guide
parameter adjustment. Activation traces from many of
these experimental reports are reprinted below (in
Figs. 9 – 11) alongside simulated activation traces of
corresponding model cells. Clearly, the model learned
faster than real monkeys. This was largely due to simulation
parameters, in particular, a lack of any uncontrolled
distracting stimuli, no noise, unflagging motivational input
signals, maximal between-task compatibility, and a conveniently high learning (weight adjustment) rate. Less
favorable parameters would amplify the number of trials to
Illustration of model cell dynamics in one task. Panels
A – U of Fig. 8 depict the dynamics of 22 model cells during
the overlap task; see Fig. 1. Panel A shows activations of
two model FEF input-layer cells whose dynamics differ
because only one cell’s receptive field is such that it (dotted
line) receives both an early input based on the target’s
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Table 4
Electrophysiological cell types
Cell type names
comparison figure
1. Visual
2. Visual with second burst
3. Visuomovement
4. Discriminating visuomovement
5. Transient Sensory-Movement
6. Presaccadic
7. Postsaccadic
8. Sustained Sensory-Movement
9. Preparatory Set/Fixation
10. Burster
11. Pause-burster
12. Burster
13. Burster
14. Pauser
15. Movement-related
16. Saccade-related
17. Fixation-related
Fig. 2A
Fig. 2A
Fig. 2B
Fig. 2B
Fig. 2C
Fig. 2D
Fig. 2E
Fig. 2F
Fig. 2G
Fig. 3A
Fig. 3A
Fig. 3B
Fig. 3C
Fig. 4A
Fig. 4C
Fig. 4B
Fig. 4D
Schall et al. (1995a)
Schall (1991)
Schall and Bichot (1998)
Motor Thalamus
retinotopic locus and a delayed input, from ITp, based on a
conjunction of the target’s position and feature. The other
cell (solid line) shares the initial input but lacks the second
because it is tuned to a different position-feature conjunction in ITp. All other panels show only one cell. Panels M
and N respectively show a winning and a losing FEF planlayer cell. The Fig. 8 caption gives the number of the
equation (see Mathematical Model) that mediated each
Reinforcement learning of plan activations. The trial-bytrial progress of successful learning and performance of the
task series was described above. The dotted lines in
Fig. 8L – N depict the second messenger traces ðFÞ
by recent plan cell activations. These intracellular trace
variables allowed the model to bridge the time delays
between plan cell offset and the later arrival at planning cells
of signals indicating rewarding or non-rewarding response
outcomes. Consequently, reinforcement learning in model
cortico-cortical synapses was able to bring plan cell
activation under control of inputs to FEF. The simulations
Schall (1991), Segraves and Goldberg (1987)
Bruce et al. (1985), Schall (1991), Segraves and Goldberg (1987)
Bruce et al. (1985), Schall (1991)
Hanes et al. (1998), Schall (1991)
Schall et al. (1995b), Schall (1991)
Turner and Anderson (1997)
Turner and Anderson (1997)
Schlag-Rey and Schlag (1984), Turner and Anderson (1997)
Wichmann et al. (1994)
Hikosaka and Wurtz (1989)
Kalaska and Crammond (1995)
Munoz and Wurtz (1995)
Munoz and Wurtz (1993)
exemplify computational hypothesis 12, that reinforcement
learning establishes stimulus control over plans, not
responses, even though plan cell activation is suppressed
by feedback inhibition from post-saccadic cells well before
reward or non-reward signals have been generated. More
generally, the model illustrates how learning and performance processes evolve in parallel and in real-time, with no
need for ‘off-line’ learning.
Comparisons of model with real cell dynamics.
Figs. 9 –11 show side-by-side comparisons of 17 model
cell activations with neurophysiological data (citations in
Table 4). These comparisons illustrate the ability of the
model to match data across the range of tasks simulated. In
particular, Fig. 9A data and simulation are from the delay
task, 9B from the discrimination task, and 9C –9G from the
overlap task. Fig. 10 data and simulation are from the delay
task, and Fig. 11 from the overlap task.
Fig. 9 shows plots of model cell behavior alongside
reprinted plots of electrophysiological data for various real
FEF cells with distinctive types of task-related dynamics.
Fig. 8. Activation dynamics of 22 simulated cell types during performance of the overlap task. From left to right in each column of plots, the three vertical lines
respectively mark target onset time, fixation light offset time, and saccade onset time. (A) Two FEF input layer cells (governed by Eq. (12) whose retinotopic
receptive fields cover the target position. Only one shows a second burst due to delayed input (from ITp) conditional upon target’s position and features. (B)
FEF layer VI cell Eq. (13). (C) GPi/SNr cell Eq. (35) associated with the Fixation-feature GCZ. (D) GPi/SNr cell Eq. (35) associated with the Target-feature
GCZ.(E) Striatal SPN in the direct pathway Eq. (27) associated with SC saccade-related cells. (F) SNr saccade-related cell Eq. (32) that receives inhibition
from the cell in (E). (G) SNr cell Eq. (32) associated with SC fixation cells. (H) Fixation cell Eq. (11) of the SC. (I) Saccade-related cell Eq. (11) of the SC. (J)
FEF post-saccadic cell Eq. (26). (K) Thalamic cell Eq. (38) associated with the Target GCZ. The late (post movement) discharge of this cell is associated with
postsaccadic fixation of the Target object. There is little pre-movement thalamic activity, because the movement to the still visible target is generated by the
reactive, PPC-SC, pathway (not via FEF) once Fixation stimulus offset removes the indirect channel’s STOP signal. (L) FEF fixation plan cell Eq. (14) of layer
variable Eq. (45). (M) Winning FEF saccade plan cell Eq. (14) of layer III/Va, showing discharge of the sustained
III/Va. Dotted line shows associated trace ðFÞ
sensory-movement type. (N) Losing FEF saccade plan cell Eq. (14) showing discharge of the transient sensory-movement type. (O) Striatal direct path SPN Eq.
(33) associated with FEF fixation cells. (P) Striatal direct path SPN Eq. (33) associated with FEF saccade-related cells. (Q) PPC fixation-related cell Eq. (6). (R)
PPC saccade-related cell Eq. (6). (S) FEF layer Vb fixation cell Eq. (21). (T) Striatal SPN of the indirect pathway Eq. (34) that inhibits the FEF Target-feature
module. This cell receives strong input from the FEF fixation cell in (L). (U) GPe cell Eq. (36) in the pathway that inhibits the FEF Target-feature module. This
cell receives strong inhibition from the cell in (T).
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Fig. 9. Comparison of dynamics of real and simulated FEF cells. Data appear above model fits in all panels except (G). Fig. 9A simulation and data are from the
delay task, 9B from the discrimination task, and 9C, D, E, F, and G from the overlap task. (A) Discharge of FEF visual cells and activation level of model FEF
input layer cells. [Data adapted with permission from Schall et al., 1995a.] (B) FEF visuomovement cells and model FEF plan layer cells. In the simulation plot,
the solid trace is from the winning plan and the dashed trace from a losing plan. [Data adapted with permission from Schall and Bichot, 1998.] (C) FEF transient
sensory-movement cells and model FEF losing plan layer cells [Data adapted with permission from Schall, 1991.] (D) FEF presaccadic movement cells and
model FEF layer Vb (output) layer cells. (E) FEF postsaccadic cells and model FEF inhibitory feedback cells [(D) and (E) data plots adapted with permission
from Schall, 1991.] (F) FEF sustained sensory-movement cells and model FEF plan layer cells that represent planned but not yet executed saccades. [Data
adapted with permission from Hanes et al., 1998.] (G) FEF preparatory set cells (left) and model FEF fixation cells (right). Vertical line on right denotes
saccade onset. [Data adapted with permission from Schall et al., 1995b.]
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Fig. 10. Comparison of dynamics of simulated and real BG and PNR–THAL cells for the delay task. (A) On left, real GPe cells increase their firing rate prior to
movement onset whereas GPi cells pause. On the right are corresponding model GPe and GPi cells. In the model, GPi pausing permits movement execution.
Model STN cell activity beginning at movement initiation transiently increases both the GPi and GPe activities. [Data adapted with permission from Turner
and Anderson, 1997.] (B) Above, real thalamic cells show a transient burst just prior to movement initiation. Below, the model thalamic burst reflects excitation
from layer 6 and transient disinhibition (gating) by the GPi. [Data adapted with permission from Schlag-Rey and Schlag, 1984.] (C) Above, real STN cells
show a transient increase in activity immediately following movement onset. Below, the model STN burst reflects a transient corollary discharge from FEF
output layer cells. [Data adapted with permission from Wichmann et al., 1994.]
Fig. 11. Comparison of dynamics of simulated and real PPC, SC, and SC-related BG cells for the overlap task. Data appear above simulation in each panel. (A)
Real SNr cell pauses during saccades near its preferred direction. Model SNr cell pauses to allow model SC to generate a saccade near the cell’s preferred
direction. [Data adapted with permission from Hikosaka & Wurtz, 1989, Fig. 10b.] (B) Real SC burst cell activity during the overlap task compared with model SC
burst cell. [Data adapted with permission from Munoz & Wurtz, 1995.] (C) Real PPC activity during a task similar to the overlap task compared with model PPC
activity during the overlap task. [Data adapted with permission from the reaching study of Kalaska & Crammond, 1995.] (D) Real fixation-cell activity in the
rostral SC, where fixation cells pause briefly during a saccade, compared to a model SC fixation cell. [Data adapted with permission from Munoz & Wurtz, 1993.]
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Fig. 9A shows FEF visual cells (Schall et al., 1995a). These
cells fire weakly at 50 ms after stimulus onset (time t ¼ 0) in
the receptive field. If the stimulus is a preferred feature for
the cell, a second burst occurs at 100 ms due to input from
ITp. In the model (see Figs. 2 and 6a), these are FEF input
layer cells (Fxyi
in Eq. (12)). Fig. 9B shows traces from two
FEF discriminating visuomovement cells (Schall & Bichot,
1998). Such cells initially respond to either a preferred or
non-preferred stimulus, but begin to discriminate with a
bifurcation around 100– 120 ms after stimulus onset. In the
model, these are plan layer cells (Fxyi
in Eq. (14)), and their
bifurcation is due to both recurrent, competitive dynamics
and the selectiveness of their inputs (shown in Fig. 9A).
Either factor may explain the slightly faster decline of
activity in the ‘losing’ model cell than in the corresponding
real cell. Fig. 9C exemplifies transient sensory-movement
cells (Schall, 1991), which fire both in response to the
stimulus and around the time of the saccade. In the model,
these are plan layer cells (Fxyi
; Eq. (14)) that lose the
competition for sustained activity, due to recurrent inhibition in (18). A corollary discharge from the SC (term Sxy
in (15)) drives the burst just before movement onset. That
the 9C real cell has some sustained activity whereas the 9C
model cell has none is not significant because both data and
model indicate a continuum from none to the robust
sustained activity shown in 9F.
Fig. 9D shows presaccadic cells (Schall, 1991), which
peak at the time of saccade onset, when activity is
sufficient to initiate a saccade but inhibition from
postsaccadic cells (Fxy
; Eq. (26)) has not yet taken
effect. In the model, these are layer Vb output cells
; Eq. (21)) that mediate execution of the saccade.
Some movement-related layer Vb cells that projected to
SC also showed visual rather than movement activity
when the stimulus in the receptive field was not to be
foveated, and others showed weak but sustained, working
memory-like activity. These were epiphenomenal in the
model and emerged due to the complex constraints on
parameters governing movement gating in layer
V. However, these unexpected findings agree with and
functionally interpret otherwise puzzling observations of
non-motor responses in SC-projecting cells in layer V of
FEF (Sommer & Wurtz, 2000). Fig. 9E shows postsaccadic cells (Schall, 1991), which begin firing at the
point of saccade initiation. In the model, these cells
; Eq. (26)), which are driven by a corollary
discharge of movement initiation from SC burst cells,
serve to shut off the successfully executed saccadic plan.
Fig. 9F shows sustained sensory-movement cells (Hanes
et al., 1998), which are active from stimulus onset until
just after saccade initiation. These contrast with preparatory set or fixation cells, which cease activity before,
rather than after, saccade initiation. In the model, the
sustained sensory-movement cells (Eq. (14)) exist in the
plan layer (II/III/Va) and represent planned but not yet
executed saccades. Their sustained activity is initiated by
afferents from the input layer (F ðIÞ in (12)), maintained
by recurrent self-excitation, and shut off by inhibition
from postsaccadic cells (F ðXÞ in (26)). Fig. 9G shows
FEF preparatory set cells (Schall et al., 1995b). The
vertical line on the right denotes saccade onset. Set cells
remain active during fixation, and shut off prior to
saccade initiation. In the model FEF, such cells exist in
both the plan (Fxyi
; Eq. (14)) and the output layers (Fxyi
Eq. (21)) to maintain fixation. They are driven by foveal
activity and are shut off by inhibition from saccadeðOÞ
related output cells Fxyi
; subject to parameter kfop in (18)
and the last term in (25), when saccade initiation is
Fig. 10 shows neurophysiological data collected from
BG and PNR –THAL cell types (see Fig. 2) and the
dynamics of corresponding simulated cells. Fig. 10A (left
side) shows data from GPe and GPi from the reaching
study of Turner and Anderson (1997). GPe cells (BðGPeÞ
Eq. (36)) increase their firing rate prior to movement
onset, whereas GPi cells (BðGPiÞ
; Eq. (35)) show a pause
prior to movement onset. Similar pausing data have been
reported by Hikosaka and Wurtz (1983) for SNr in a
saccade task, as shown in Fig. 11. In the simulation
(Fig. 10A, right), the GPi pause is due to increased
inhibition from both the GPe (BðGPeÞ
; Eq. (36)) and the
striatum (BðSDÞ
; Eq. (33)), and the pause disinhibits the
PNR-THAL, thereby allowing the movement to be
executed. STN activity ðBðSTNÞ ; Eq. (37)) that begins at
movement initiation transiently increases both the GPi
and GPe activities. Fig. 10B shows PNR-THAL cells
(Schlag-Rey & Schlag, 1984; Eq. (38)) that show a
transient burst just prior to movement initiation. In the
model, this reflects excitation from FEF layer VI ðFiðGÞ Þ
and transient disinhibition (gating) in the GPi-thalamic
projection. Fig. 10C shows STN cells (Wichmann,
Bergman, & DeLong, 1994) that exhibit a transient
increase in activity immediately following movement
onset. In the model, this reflects a transient corollary
discharge from FEF output layer cells (Fxyi
; Eq. (21)) to
the STN ðB
; Eq. (37)).
Fig. 11 shows the dynamics of PPC, SC, and SCrelated BG cells. Fig. 11A shows an SNr cell pausing
prior to saccade initiation. In the simulation (GðSNrÞ
xy ; Eq.
(32)), the similar pause is caused by inhibitory action of
the model direct path from the striatum (GðSDÞ
xy ; Eq. (27)).
Indeed, the mirror image of the SNr off-response, also
shown in Fig. 8F, can be seen in the striatal on-response
shown in Fig. 8E, consistent with data reviewed in
Hikosaka and Wurtz (1989). Fig. 11B shows SC burst
cell activity (Munoz & Wurtz, 1995) during the overlap
task. In the simulation (Sxy ; Eq. (11)), the excitatory
inputs that energize the burst come from both PPC cells
(Pxy) and FEF output cells ðFxyi
Þ: Although the former
excite the SC throughout the overlap interval, the SC
shows a burst response only after the removal of
inhibition from the model SNr ðGðSNrÞ
xy Þ: Fig. 11C shows
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
PPC activity (Kalaska & Crammond, 1995) during a
GO –NOGO task on a GO trial similar to an overlap task
trial. The real and simulated PPC activities show
sustained activity during the delay interval followed by
a pre-movement burst response. However, note that the
model lacks the tonic baseline activity seen in the real
cell. The model (Pxy ; Eq. (6)) attributes the sustained
delay-interval activity to inputs from visual cortex ðIxy
and the FEF planning layer III/Va ðFxyi Þ; and the burst
response to input from the FEF output layer Vb ðFxyi
Fig. 11D shows fixation-related activity in the SC
(Munoz & Wurtz, 1993). Such SC fixation cells pause
briefly during a saccade. In the simulation (Sxy ; Eq. (11)),
the pause results from both the collapse of excitatory
input from FEF fixation-related output cells ðF11i
Þ and
the onset of inhibition from the fixation-related component of the SNr ðGðSNrÞ
11 Þ:
10. Discussion
General issues. The theory developed in this paper
proposes how the BG interact with laminar circuits in the
frontal cortex and SC to help satisfy the staging requirements of conditional voluntary behavior. Fourteen
functional hypotheses were elaborated to clarify the roles
played by components of the known circuitry treated by the
theory. The mathematical model based on this theory is able
to account for a wide range of anatomical, neurophysiological, and psychophysical data about planned and reactive
saccadic eye movements. During performance, the model
regenerated seventeen distinct types of activation dynamics
exhibited by electrophysiologically-recorded cell types,
while also predicting the behavior of several additional
cell types. By virtue of learning guided by reinforcement
signals, and selective cue sensitivities of premotor mapping
zones, the model system was able to learn five, and to
perform seven, oculomotor tasks, without forgetting previously learned tasks, and with a single set of parameters. As
such, the TELOS model goes some distance toward
explaining both procedural learning of arbitrary conditionaction rules (Wise & Murray, 2000) and the ability to switch
at will among the large number of rules that may eventually
be learned and simultaneously preserved in memory.
Comparison with other models. The model introduces
new concepts while also incorporating and extending
concepts from previous models. The concept of zones
consisting of cortex and BG channels has been discussed
previously (Beiser & Houk, 1998; Houk & Wise, 1995;
Redgrave et al., 1999) in terms of cortico-BG modules.
Consistent with data on distinct sensitivities of frontal
premotor areas (review in Passingham, 1993), we have
extended the module concept to denote groups of cells
organized into gatable cortical zones (GCZs) that each
mediate a distinctive sensorimotor strategy or mapping. The
resultant GCZ concept shares features with the ‘mixture of
experts’ model (Jacobs, Jordan, Nowland, & Hinton, 1991),
which has been discussed with references to BG function
(Graybiel, Aosaki, Flaherty, & Kimura, 1994; Graybiel,
1998). However, whereas previously the striatum has been
suggested as the location of the ‘experts’ (Graybiel, 1998),
in the present model of primate action control, the ‘experts’
are cortical, indeed frontal, and the BG function as a GCZ
BG control of cortico-thalamo-cortical projections.
Previous models have assumed reciprocal excitation
between the cortex and thalamus (Arbiband Dominey,
1995; Beiser & Houk, 1998; Frank, Loughry, & O’Reilly,
2001; Houk & Wise, 1995; Taylor & Taylor, 2000), which
was proposed as a basis for sustained, working memory-like
activity enabled by BG disinhibition of the thalamus. The
working memory in turn biases, rather than determines,
movement selection (Houk & Wise, 1995). Studies of the
laminar anatomy of frontal cortex show thalamocortical
projections terminating on cells in layers III and V, but not
layer VI (Iriki, Pavlides, Keller, & Asanuma, 1991;
Zin-Ka-Ieu, Roger, & Arnault, 1998). Since layer VI is
the main source of corticothalamic projections, its apparent
omission as a recipient of thalamocortical afferents
challenges the cortico-thalamo-cortical loop theory of
working memory storage. A reciprocal thalamocortical
excitation remains possible if layer III cells excite layer VI,
or if thalamocortical fibers terminate on dendrites of layer
VI neurons that extend into overlying laminae, but the
known thalamic excitation of layer V will cause outputs that
would—on the loop theory—make it impossible for a single
cortical area to function both to reverberate a plan during a
delay interval and then to also execute that plan at the end of
the delay. In our theory, both functions are realized by a
single cortical area. Given the anatomical constraints and
the ease with which basic working memory functionality
can be achieved by local circuits, or even by intracellular
mechanisms, our theory treats plan maintenance during
delays as a local circuit property and treats BG-mediated
disinhibition of thalamus as a mechanism for gating
excitation of layer V and, thereby, execution of action
plans. Finally, it is worth noting that nothing in our
interpretation precludes a role for the BG in selective
activation of prefrontal areas supporting working memory.
Lateral vs. feedforward inhibition within the striatum.
Recurrent competition between striatal SPNs, mediated by
potent lateral inhibition among SPNs, has been assumed by
previous models (Arbib& Dominey, 1995; Beiser & Houk,
1998; Contreras-Vidal & Schultz, 1999). However, other
data (Jaeger et al., 1994) challenge the potency assumption.
Also, it is difficult to reconcile the idea of many actively
competing SPNs with observations of a relatively silent
striatum. The present model resolves the dilemma by
emphasizing the competitive role of the GABA-SIs
(GABA-ergic striatal interneurons), which, though outnumbered by SPNs 20 to 1, mediate potent feedforward
inhibition (Koos & Tepper, 1999; Wilson et al., 1989). In
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
our model, such cells prevent disparate plans from gaining
simultaneous control of the same executive degrees of
freedom. Although the model GABA –SIs mediate striatumlevel competition between vying plans, choice-making
cannot be wholly localized to the striatum. The present
model broadly distributes plan-selective dynamics across
competitive inhibitory mechanisms in the thalamus, SC, and
both supragranular and infragranular cortical layers. Finally,
the existence of striatal recurrent inhibition is compatible
with the present model. Although not needed to make the
selection, recurrent inhibition would be useful in vivo to
help sustain the striatal choice in the face of noise and
modest input fluctuations (which were not present in our
deterministic simulations).
BG direct and indirect pathways. The few BG computational models that treat the indirect pathway have ascribed
to it roles other than a STOP function. In some models, the
indirect pathway generally opposes the direct pathway
effect, without explicit justification for why this is needed
(Contreras-Vidal & Stelmach, 1995; Suri et al., 1997; Suri,
Bargas, & Arbib, 2001). Gurney, Prescott, & Redgrave
(2001a,b) suggested that the components of the indirect
pathway may be thought of as forming a ‘control’ pathway,
which modulates the tonic arousal of the GPi in proportion
to the overall level of dopamine and excitation of the
striatum. Berns and Sejnowski (1998) assumed that
reciprocal activity between the STN and GPe mediated a
kind of working memory which subserved sequence
production. Contreras-Vidal and Stelmach (1995) modeled
the GPe’s possible role in enhancing the effects of dopamine
loss in Parkinson’s disease. Our model reconceptualizes the
indirect pathway and suggests a specific rationale for why it
is needed in addition to the direct pathway, namely to
actively STOP inappropriate action plans from execution
until such time as they become appropriate.
Dominey and Arbib model. Arbib and Dominey (1995)
and Dominey et al. (1995) reported a model of the FEF, BG,
and related oculomotor areas, with which our model shares
several features. Their model also used retinotopic and motor
error map representations throughout. Their FEF model
contained visual, memory, fixation, and saccade ‘layers’,
which although not identified with cortical laminae, resemble
the cortical layers of the present model. Their visual layer
was similar to our visual or input layer, except that our visual
cells do not drive FEF output (output) cells directly. Their
FEF memory layer corresponds to saccade-related cells in
our plan layer, and their ‘fixation ON’ layer corresponds to
fixation-related cells in our plan layer (both here predicted to
reside in cortical layers II/III/Va). Their FEF output layer
corresponds to our output layer (here predicted to reside in
layer Vb), in that both drive the SC, although their output
layer does not receive corticothalamic projections, which
enable gating of movement execution in our model. Their
model also had distinct FEF- and SC-controlling BG
channels and assumed a motor error map representation in
nigro-SC projections, as does ours. The two models address a
large and overlapping body of data.
Additional important features distinguish the two
models. Our explicit treatment of the laminar anatomy of
frontal cortex led us to model the BG as a system for gating
plan execution rather than for reverberation of working
memories. Our model relies on neither striatal lateral
inhibition nor reciprocal thalamocortical excitation.
Whereas the Dominey et al. (1995) model simulates
sequence learning, our model simulates multiple task
learning and, during performance of the multiple tasks,
simulates the dynamics of seventeen electrophysiologicallyidentified cell types spread across all the brain regions
treated. Their model predicts that corticostriatal projections
mediate conditional motor learning, whereas our model
predicts that corticocortical projections from anterior or
posterior IT to the FEF plan layers (II/III/Va) mediate
conditional motor learning, including ‘What-to-Where’
transformations. Our model also implicates the indirect
channel in the conditioned STOP function neglected in prior
models. Their model predicts that working memory activity
is shut off by a nonspecific, inhibitory corollary discharge
from the SC to the thalamus. Ours proposes that the SC
drives postsaccadic cells in cortex to shut off specific,
already-executed, movement plans.
Future extensions. The TELOS model may be extended
in several ways. Its treatment of the ability of the frontal
cortex to replace a parietal target representation excited by a
transient visual input with a remembered target plan, and to
generate an internally guided saccade following resolution
of the parietal competition, applies with minor changes to
how primates perform the anti-saccade task (Amador,
Schlag-Rey, & Schlag, 1998; Everling, Spantekow, Krappmann, & Flohr, 1998; Funahashi, Chafee, & Goldman-Rakic, 1993; Hallett, 1978). One of these changes would be
the addition of the SEFs to expand the representational
scope (Chen & Wise, 1995a, 1995b) of the model’s
premotor GCZs. More generally, although we have focused
primarily on the FEF, BG, and SC, the anatomical principles
used to structure the model govern many other parts of the
frontal cortex (e.g. Middleton & Strick, 2000). Therefore the
model should generalize to other frontal cortical areas. For
another example, the prefrontal cortex (PFC, e.g. areas 9
and 46) is represented in the model as a single layer working
memory (Fig. 2) which could itself be expanded into a
multi-layered system similar to the present FEF model. The
output layer (or even plan layer) of such an expanded
prefrontal model could provide the control signals to the
FEF layer VI. This would constitute a kind of hierarchical
decision-making scheme, in which the PFC decides, based
on current working memory activity, what strategies will be
primed in lower areas such as FEF. These lower areas then
determine the specifics of a planned movement.
Another fruitful direction for extensions is the explanation
of serial behavior. In the existing model, if a ‘losing’ plan
remains, after the winning plan is executed and has its
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
activity deleted by postsaccadic cell activity, then the losing
plan may be executed subsequently. This means that the
model can serve as part of what has been called a competitive
queuing system, in which several to-be-performed plans are
simultaneously active before movement begins. Then
through iterative choice making (and deletion of plans as
they are executed), the corresponding actions are executed in
a sequence determined by the initial gradient of activation
levels, namely from most to least active. Evidence for such
gradient representations of winning versus losing plans, and
of planned sequences, have recently been reported in
prefrontal and dorsal premotor cortices (Averbeck et al.,
2002; Cisek & Kalaska, 2002). This mechanism can be used
to help learn and perform movement sequences (Boardman
& Bullock, 1991; Bullock & Rhodes, 2003; Grossberg, 1982;
Grossberg & Kuperstein, 1986; Hartley & Houghton, 1996),
for which the BG system has been implicated (e.g. Aldridge
& Berridge, 1998; Kermadi, Jurquet, Arzi, & Joseph, 1993;
Martin, Phillips, & Iansek, 1994; Weiss, Stelmach, & Hefter,
J.B. was supported in part by Defense Advanced
Research Projects Agency and the Office of Naval Research
(ONR N00014-95-1-0409, ONR N00014-92-J-1309, and
ONR N00014-95-1-0657). D.B. was supported in part by
Defense Advanced Research Projects Agency and the Office
of Naval Research (ONR N00014-95-1-0409, ONR
N00014-92-J-1309) and the National Institute of Mental
Health (R01 DC02852). S.G. was supported in part by Air
Force Office of Scientific Research (AFOSR F49620-01-10397), Defense Advanced Research Projects Agency and
the Office of Naval Research (ONR N00014-95-1-0409,
ONR N00014-92-J-1309, ONR N00014-95-1-0657, ONR
N00014-01-1-0624), and the National Science Foundation
(NSF IRI-97-20333).
Albin, R. L., Young, A. B., & Penney, J. B. (1989). The functional anatomy
of basal ganglia disorders. Trends in Neurosciences, 12, 366– 374.
Aldridge, J. W., & Berridge, K. C. (1998). Coding of serial order by
neostriatal neurons: A natural action approach to movement sequence.
Journal of Neuroscience, 18, 2777–2787.
Alexander, G., & Crutcher, M. (1990). Functional architecture of basal
ganglia circuits: neural substrates of parallel processing. Trends in
Neurosciences, 13, 266–271.
Amador, N., Schlag-Rey, M., & Schlag, J. (1998). Primate antisaccades.
I. Behavioral characteristics. Journal of Neurophysiology, 80, 1775–1786.
Amos, A. (2000). A computational model of information processing in the
frontal cortex and basal ganglia. Journal of Cognitive Neuroscience, 12,
Arbib, M. A., & Dominey, P. F. (1995). Modeling the roles of basal ganglia in
timing and sequencing saccadic eye movements. In J. Houk, J. Davis,
& D. Beiser (Eds.), Models of information processing in the basal
ganglia (pp. 149–162). Cambridge, MA: MIT Press.
Asaad, W. F., Rainer, G., & Miller, E. K. (2000). Task-specific neural
activity in the primate prefrontal cortex. Journal of Neurophysiology,
84, 451–459.
Averbeck, B. B., Chafee, M. V., Crowe, D. A., & Georgopoulos, A. P.
(2002). Parallel processing of serial movements in prefrontal cortex.
Proceedings of the National Academy of Science, 99, 13172–13177.
Bao, S., Chan, V. T., & Merzenich, M. M. (2001). Cortical remodelling
induced by activity of ventral tegmental dopamine neurons. Nature.,
412, 79–83.
Bar, M., Tootell, R. B. H., Schacter, D. L., Greve, D. N., Fischl, B., Mendola,
J. D., Rosen, B. R., & Dale, A. M. (2001). Cortical mechanisms specific
to explicit object recognition. Neuron, 29, 529 –535.
Barbas, H. (1988). Anatomic organization of basoventral and mediodorsal
visual recipient prefrontal regions in the rhesus monkey. Journal of
Comparative Neurology, 276, 313–342.
Barbas, H., & Mesulam, M. M. (1981). Organization of afferent input to
subdivisions of area 8 in the rhesus monkey. Journal of Comparative
Neurology, 200, 407–431.
Basso, M. A., & Wurtz, R. H. (1998). Modulation of neuronal activity in
superior colliculus by changes in target probability. Journal of
Neuroscience, 18, 7519–7534.
Bayer, H. M., Handel, A., & Glimcher, P. W. (1999). Do the saccaderelated neurons of the substantia nigra pars reticulata (SNr) carry
information about saccadic amplitude or endpoint? Society for
Neuroscience Abstracts, 25, 1921.
Beiser, D. G., & Houk, J. C. (1998). Model of cortical-basal ganglionic
processing: Encoding the serial order of sensory events. Journal of
Neurophysiology, 79, 3168–3188.
Berendse, H. W., de Galis, G. Y., & Groenewegen, H. J. (1992).
Topographical organization and relationship with ventral striatal
compartments of prefrontal corticostriatal projections in the rat. Journal
of Comparative Neurology, 316, 314–347.
Berger, B., Trottier, S., Verney, C., Gaspar, P., & Alvarez, C.
(1988). Regional and laminar distribution of the dopamine and serotonin
innervation in the macaque cerebral cortex: a radioautographic study.
Journal of Comparative Neurology, 273, 99– 119.
Berns, G., & Sejnowski, T. (1998). A computational model of how the basal
ganglia produce sequences. Journal of Cognitive Neuroscience, 10,
108– 121.
Berretta, S., Parthasarathy, H. B., & Graybiel, A. M. (1997). Local release
of GABAergic inhibition in the motor cortex induces immediate-early
gene expression in indirect pathway neurons of the striatum. Journal of
Neuroscience, 17, 4752–4763.
Bichot, N. P., & Schall, J. D. (1999). Effects of similarity and history on
neural mechanisms of visual selection. Nature Neuroscience, 2,
549– 554.
Bichot, N. P., Schall, J. D., & Thompson, K. G. (1996). Visual feature
selectivity in frontal eye fields induced by experience in mature
macaques. Nature, 381, 697–699.
Boardman, I., & Bullock, D. (1991). A neural network model of serial order
recall from short-term memory. Proceedings of the International Joint
Conference on Neural Networks, II, 879–884.
Brown, J., Bullock, D., & Grossberg, S. (1999). How the basal ganglia use
parallel excitatory and inhibitory learning pathways to selectively
respond to unexpected rewarding cues. Journal of Neuroscience, 19,
Brown, J., Bullock, D., & Grossberg, S. (2000). How laminar frontal cortex
and basal ganglia circuits interact to control planned and reactive
saccades. Society for Neuroscience Abstracts, 26.
Brown, L. L., Makman, M. H., Wolfson, L. I., Dvorkin, B., Warner, C., &
Katzman, R. (1979). A direct role of dopamine in the rat subthalamic
nucleus and an adjacent intrapeduncular area. Science, 206,
Bruce, C. J., Goldberg, M. E., Bushnell, M. C., & Stanton, G. B. (1985).
Primate frontal eye fields. II. Physiological and anatomical correlates of
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
electrically-evoked eye movements. Journal of Neurophysiology, 54,
714– 734.
Bullier, J., Schall, J. D., & Morel, A. (1996). Functional streams in occipitofrontal connections in the monkey. Behavioral Brain Research, 76,
Bullock, D., & Grossberg, S. (1991). Adaptive neural networks for control
of movement trajectories invariant under speed and force rescaling.
Human Movement Science, 10, 3–53.
Bullock, D., & Rhodes, B. (2003). Competitive queuing for serial planning
and performance. In M. Arbib (Ed.), (2nd ed.) (pp. 241 – 244).
Handbook of brain theory and neural networks, Cambridge, MA:
MIT Press.
Burman, D. D., & Bruce, C. J. (1997). Suppression of task-related saccades
by electrical stimulation in the primate’s frontal eye field. Journal of
Neurophysiology, 77, 2252– 2267.
Butler, A. B., & Hodos, W. (1996). Comparative vertebrate neuroanatomy.
New York: Wiley-Liss.
Calabresi, P., Gubellini, P., Centonze, D., Picconi, B., Bernardi, G.,
Chergui, K., Svenningsson, P., Fienberg, A. A., & Greengard, P.
(2000). Dopamine and cAMP-regulated phosphoprotein 32 kDa
controls both striatal long-term depression and long-term potentiation, opposing forms of synaptic plasticity. Journal of Neuroscience, 20, 8443–8451.
Canteras, N. S., Shammah-Lagnado, S. J., Silva, B. A., & Ricardo, J. A.
(1990). Afferent connections of the subthalamic nucleus: a combined
retrograde and anterograde horseradish peroxidase study in the rat.
Brain Research, 513, 43–59.
Cavada, C., & Goldman-Rakic, P. S. (1991). Topographic segregation of
corticostriatal projections from posterior parietal subdivisions in the
macaque monkey. Neuroscience, 42, 683–696.
Chafee, M. V., & Goldman-Rakic, P. S. (2000). Inactivation of parietal
and prefrontal cortex reveals interdependence of neural activity
during memory-guided saccades. Journal of Neurophysiology, 83,
Charpier, S., & Deniau, J. M. (1997). In vivo activity-dependent plasticity
at cortico-striatal connections: evidence for physiological long-term
potentiation. Proceedings of the National Academy of Science, 94,
Chen, L. L., & Wise, S. P. (1995a). Neuronal activity in the supplementary
eye field during acquisition of conditional oculomotor associations.
Journal of Neurophysiology, 73, 1101–1121.
Chen, L. L., & Wise, S. P. (1995b). Supplementary eye field contrasted with
the frontal eye field during acquisition of conditional oculomotor
associations. Journal of Neurophysiology, 73, 1122–1134.
Cisek, P., & Kalaska, J. F. (2002). Simultaneous encoding of multiple
potential reach directions in dorsal premotor cortex. Journal of
Neurophysiology, 87, 1149– 1154.
Cohen, J. D., Braver, T. S., & Brown, J. W. (2002). Computational
perspectives on dopamine function in prefrontal cortex. Current
Opinion in Neurobiology, 12, 223–229.
Contreras-Vidal, J., & Schultz, W. (1999). A predictive reinforcement
model of dopamine neurons for learning approach behavior. Journal of
Computational Neuroscience, 6, 191 –214.
Contreras-Vidal, J., & Stelmach, G. E. (1995). A neural model of basal
ganglia-thalamocortical relations in normal and parkinsonian movement. Biological Cybernetics, 73, 467–476.
Cowan, N. (2001). The magical number 4 in short-term memory: A
reconsideration of mental storage capacity. Behavioral and Brain
Sciences, 24, 87–185.
Crosson, B. (1985). Subcortical functions in language: a working model.
Brain and Language, 25, 257–292.
De Las Heras, S., Mengual, E., Velayos, J. L., & Gimenez-Amaya, J. M.
(1998). Re-examination of topographic distribution of thalamic neurons
projecting to the caudate nucleus. A retrograde labeling study in the cat.
Neuroscience Research, 31, 283–293.
Deng, S. Y., Goldberg, M. E., Segraves, M. A., Ungerleider, L. G., &
Mishkin, M. (1986). The effect of unilateral ablation of the frontal
eye fields on saccadic performance in the monkey. In E. Keller, &
D. Zee (Eds.), Advances in the Biosciences (Vol. 57).
Adaptive processes in visual and oculomotor systems, New York:
Dominey, P., Arbib, M., & Joseph, J. (1995). A model of corticostriatal
plasticity for learning oculomotor associations and sequences. Journal
of Cognitive Neuroscience, 7, 311 –336.
Durstewitz, D., Seamans, J. K., & Sejnowski, T. J. (2000). Dopaminemediated stabilization of delay-period activity in a network model of
prefrontal cortex. Journal of Neurophysiology, 83, 1733– 1750.
Everling, S., Spantekow, A., Krappmann, P., & Flohr, H. (1998). Eventrelated potentials associated with correct and incorrect responses in a
cued antisaccade task. Experimental Brain Research, 118, 27 –34.
Ewert, J. P., Schurg-Pfeiffer, E., & Schwippert, W. W. (1996). Influence of
pretectal lesions on tectal responses to visual stimulation in anurans:
field potential, single neuron and behavior analyses. Acta Biologica
Hungarica, 47, 89 –111.
Eyny, Y. S., & Horvitz, J. C. (2003). Opposing roles of D1 and D2 receptors
in appetitive conditioning. Journal of Neuroscience, 23, 1584–1587.
Fiala, J., Grossberg, S., & Bullock, D. (1996). Metabotropic glutamate
receptor activation in cerebellar Purkinje cells as substrate for adaptive
timing of the classically conditioned eye-blink response. Journal of
Neuroscience, 16, 3760–3774.
Fischer, B., & Weber, H. (1993). Express saccades and visual attention.
Behavioral and Brain Sciences, 16, 553 –610.
Flaherty, A. W., & Graybiel, A. M. (1991). Corticostriatal transformations
in the primate somatosensory system. Projections from physiologically
mapped body-part representations. Journal of Neurophysiology, 66,
Frank, M. J., Loughry, B., & O’Reilly, R. C. (2001). Interactions between
the frontal cortex and basal ganglia in working memory: A
computational model. Cognitive, Affective, and Behavioral Neuroscience, 1, 137–160.
Funahashi, S., Chafee, M. V., & Goldman-Rakic, P. S. (1993). Prefrontal
neuronal activity in rhesus monkeys performing a delayed anti-saccade
task. Nature, 365, 753–756.
Gancarz, G., & Grossberg, S. (1999). A neural model of saccadic eye
movement control explains task-specific adaptation. Vision Research,
39, 3123–3143.
Gaspar, P., Bloch, B., & Le Moine, C. (1995). D1 and D2 receptor gene
expression in the rat frontal cortex: cellular localization in different
classes of efferent neurons. European Journal of Neuroscience, 7,
Gerfen, C., & Wilson, C. (1996). The basal ganglia. In L. W. Swanson, A.
Bjorklund, & T. Hokfelt (Eds.), Handbook of chemical neuroanatomy
(Vol. 12) (pp. 371–468). Integrated systems of the CNS, Part III,
Holland: Elsevier Science B.V.
Gernert, M., Hamann, M., Bennay, M., Loscher, W., & Richter, A. (2000).
Deficit of striatal parvalbumin-reactive GABAergic interneurons and
decreased basal ganglia output in a genetic rodent model of idiopathic
paroxymal dystonia. Journal of Neuroscience, 20, 7052–7058.
Giuffrida, R., Volsi, G. L., Maugeri, G., & Perciavalle, V. (1985).
Influences of the pyramidal tract on the subthalamic nucleus in the cat.
Neuroscience Letters, 54, 231 –235.
Goldman-Rakic, P. S. (1987). Circuitry of primate prefrontal cortex and
regulation of behavior by representational memory. In V. B. Mountcastle, F. Plum, & S. R. Geiger (Eds.), (Vol. I) (pp. 373 –417).
Handbook of physiology, New York: Oxford University Press.
Goldman-Rakic, P. S. (1995). Toward a circuit model of working memory
and the guidance of voluntary motor action. In J. Houk, J. Davis, & D.
Beiser (Eds.), Models of information processing in the basal ganglia
(pp. 131 –148). Cambridge, MA: MIT Press.
Gnadt, J. W., & Andersen, R. A. (1988). Memory related motor planning
activity in posterior parietal cortex of macaque. Experimental Brain
Research, 70, 216–220.
Graybiel, A. M. (1998). The basal ganglia and chunking of action repertoires.
Neurobiology of Learning and Memory, 70, 119–136.
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Graybiel, A. M., Aosaki, T., Flaherty, A., & Kimura, M. (1994). The basal
ganglia and adaptive motor control. Science, 265, 1826–1831.
Gross, C. G., Desimone, R., Albright, T. D., & Schwartz, E. L. (1985).
Inferior temporal cortex and pattern recognition. Experimental Brain
Research, Supplement, 11, 179– 201.
Grossberg, S. (1971). On the dynamics of operant conditioning. Journal of
Theoretical Biology, 33, 225– 255.
Grossberg, S. (1973). Contour enhancement, short term memory, and
constancies in reverberating neural networks. Studies in Applied
Mathematics, 52, 213 –257.
Grossberg, S. (1978a). A theory of human memory: self-organization and
performance of sensory-motor codes, maps, and plans. On Progress in
Theoretical Biology, Vol. 5 (R. Rosen and F. Snell, eds), pp. 233–374.
New York: Academic Press.
Grossberg, S. (1978b). Behavioral contrast in short term memory: serial
binary memory models or parallel continuous memory models. Journal
of Mathematical Psychology, 3, 199–219.
Grossberg, S. (1982a). Processing of expected and unexpected events
during conditioning and attention: a psychophysiological theory.
Psychological Review, 89, 529 –572.
Grossberg, S. (1982b). Studies of mind and brain. Boston: Kluwer/Reidel
Grossberg, S., & Kuperstein, M. (1986). Neural dynamics of adaptive
sensory-motor control: Ballistic eye movements. Holland: Elsevier.
Grossberg, S., & Merrill, J. (1992). A neural network model of adaptively
timed reinforcement learning and hippocampal dynamics. Cognitive
Brain Research, 1, 3–38.
Grossberg, S., Roberts, K., Aguilar, M., & Bullock, D. (1997). A neural
model of multimodal adaptive saccadic eye movement control by
superior colliculus. Journal of Neuroscience, 17, 9706– 9725.
Grossberg, S., & Schmajuk, N. A. (1987). Neural dynamics of
attentionally modulated Pavlovian conditioning: Conditioned reinforcement, inhibition and opponent processing. Psychobiology, 15,
Guitton, D., Buchtel, H. A., & Douglas, R. M. (1985). Frontal lobe lesions
in man cause difficulties in suppressing reflexive glances and in
generating goal-directed saccades. Experimental Brain Research, 58,
Gurden, H., Takita, M., & Jay, T. M. (2000). Essential role of D1 but not D2
receptors in the NMDA receptor-dependent long-term potentiation at
hippocampal-prefrontal cortex synapses in vivo. Journal of Neuroscience, 20, RC106.
Gurney, K., Prescott, T. J., & Redgrave, P. (2001a). A computational model
of action selection in the basal ganglia: A new functional anatomy.
Biological Cybernetics, 84, 401 –410.
Gurney, K., Prescott, T. J., & Redgrave, P. (2001b). A computational model
of action selection in the basal ganglia: Analysis and simulation of
behavior. Biological Cybernetics, 84, 411–423.
Hall, W. C., & Lee, P. (1993). Interlaminar connections of the superior
colliculus in the tree shrew. I. The superficial gray layer. Journal of
Comparative Neurology, 332, 213–223.
Hallett, P. E. (1978). Primary and secondary saccades to goals defined by
instructions. Vision Research, 18, 1279–1296.
Hanes, D. P., Patterson, W. F. I. I., & Schall, J. D. (1998). Role of frontal
eye fields in countermanding saccades: Visual, movement, and fixation
activity. Journal of Neurophysiology, 79, 817–834.
Hanes, D. P., & Schall, J. D. (1996). Neural control of voluntary movement
initiation. Science, 274, 427 –430.
Handel, A., & Glimcher, P. W. (2000). Contextual modulation of substantia
nigra pars reticulata neurons. Journal of Neurophysiology, 83,
Handel, A., & Glimcher, P. W. (1999). Quantitative analysis of substantia
nigra pars reticulata activity during a visually guided saccade task.
Journal of Neurophysiology, 82, 3458–3475.
Hartley, T. A., & Houghton, G. (1996). A linguistically constrained model
of short-term memory for nonwords. Journal of Memory and Language,
35, 1– 31.
Hassani, O. K., Moroux, M., & Feger, J. (1996). Increased subthalamic
neuronal activity after nigral dopaminergic lesion independent of
disinhibition via the globus pallidus. Neuroscience, 72, 105–115.
Hazrati, L. N., Parent, A., Mitchell, S., & Haber, S. N. (1990). Evidence for
interconnections between the two segments of the globus pallidus in
primates: a PHA-L anterograde tracing study. Brain Research, 533,
171– 175.
Hemart, N., Daniel, H., Jaillard, D., & Crepel, F. (1995). Receptors and
second messengers involved in long-term depression in rat cerebellar
slices in vitro: a reappraisal. European Journal of Neuroscience, 7,
45– 53.
Hikosaka, O., & Wurtz, R. H. (1983). Visual and oculomotor functions
of monkey substantia nigra pars reticulata. IV. Relation of substantia
nigra to superior colliculus. Journal of Neurophysiology, 49,
Hikosaka, O., Sakamoto, M., & Usui, S. (1989). Functional properties of
monkey caudate neurons I. Activities related to saccadic eye movements. Journal of Neurophysiology, 61, 780–798.
Hikosaka, O., & Wurtz, R. H. (1989). The basal ganglia. In R. Wurtz, & M.
Goldberg (Eds.), The neurobiology of sacccadic eye movements (pp.
257– 281). Amsterdam: Elsevier.
Hodgkin, A. L. (1964). The conduction of the nervous impulse. Springfield,
IL: Thomas.
Hoesen, G. W. V., Yeterian, E. H., & Lavizzo-Mourey, R. (1981).
Widespread corticostriate projections from temporal cortex of the
rhesus monkey. Journal of Comparative Neurology, 199, 205 –219.
Horak, F. B., & Anderson, M. E. (1984). Influence of globus pallidus on
arm movements in monkeys, II. Effects of stimulation. Journal of
Neurophysiology, 52, 305– 322.
Horak, F. B., Shupert, C. L., & Mirka, A. (1989). Components of postural
dyscontrol in the elderly: A review. Neurobiology of Aging, 10,
727– 738.
Hoshi, E., Shima, K., & Tanji, J. (1998). Task-dependent selectivity of
movement-related neuronal activity in the primate prefrontal cortex.
Journal of Neurophysiology, 80, 3392–3397.
Houk, J., Adams, J., & Barto, A. (1995). A model of how the basal ganglia
generate and use neural signals that predict reinforcement. In J. Houk, J.
Davis, & D. Beiser (Eds.), Models of information processing in the
basal ganglia (pp. 249–270). Cambridge, MA: MIT Press.
Houk, J. C., & Wise, S. P. (1995). Distributed modular architectures linking
basal ganglia, cerebellum, and cerebral cortex: Their role in planning
and controlling action. Cerebral Cortex, 2, 95–110.
Humphrey, D. R., & Reed, D. J. (1983). Separate cortical systems for
control of joint movement and joint stiffness: Reciprocal activations and
coactivation of antagonist muscles. In J. E. Desmedt (Ed.), Motor
control mechanisms in health and disease. (Advances in Neurology, 39,
347– 372). New York: Raven Press.
Hutsler, J. J., & Chalupa, L. M. (1991). Substance P immunoreactivity
identifies a projection from the cat’s superior colliculus to the principal
tectorecipient zone of the lateral posterior nucleus. Journal of
Comparative Neurology, 312, 379–390.
Iriki, A., Pavlides, C., Keller, A., & Asanuma, H. (1991). Long-term
potentiation of thalamic input to the motor cortex induced by
coactivation of thalamocortical and corticocortical afferents. Journal
of Neurophysiology, 65, 1435–1441.
Iwabuchi, A., & Kubota, K. (1998). Laminar organization of neuronal
activities in area 8 of rhesus monkeys during a symmetrically
reinforced visual go/no-go task. International Journal of Neuroscience, 94, 1–25.
Jacobs, R. A., Jordan, M. I., Nowland, S. J., & Hinton, G. E. (1991). Adaptive
mixtures of local experts. Neural Computation, 3, 79–87.
Jaeger, D., Kita, H., & Wilson, C. J. (1994). Surround inhibition among
projection neurons is weak or nonexistent in the rat neostriatum.
Journal of Neurophysiology, 72, 2555–2558.
Jones, E. G., Coulter, J. D., Burton, H., & Porter, R. (1977). Cells of origin and
terminal distribution of corticostriatal fibers arising in the sensory-motor
cortex of monkeys. Journal of Comparative Neurology, 173, 53–80.
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Kalaska, J. F., & Crammond, D. J. (1995). Deciding not to GO:
neuronal correlates of response selection in a GO/NOGO task in
primate premotor and parietal cortex. Cerebral Cortex, 5, 410–428.
Kang, Y., & Kayano, F. (1994). Electrophysiological and morphological
characteristics of layer VI pyramidal cells in the cat motor cortex.
Journal of Neurophysiology, 72, 578–591.
Kawaguchi, Y. (1997). Neostriatal cell subtypes and their functional roles.
Neuroscience Research, 27, 1–8.
Kawaguchi, Y., Wilson, C. J., Augood, S. J., & Emson, P. C. (1995).
Striatal interneurones: chemical, physiological and morphological
characterization. Trends in Neurosciences, 18, 527 –535.
Kemel, M. L., Desban, M., Gauchy, C., Glowinski, J., & Besson, M. J.
(1988). Topographical organization of efferent projections from the cat
substantia nigra pars reticulata. Brain Research, 455, 307–323.
Kermadi, I., Jurquet, Y., Arzi, M., & Joseph, J. P. (1993). Neural activity in
the caudate nucleus of monkeys during spatial sequencing. Experimental Brain Research, 94, 352–356.
Kimura, A., Caria, M. A., Melis, F., & Asanuma, H. (1994). Long-term
potentiation within the cat motor cortex. Neuroreport, 5, 2372– 2376.
Kita, H., Kosaka, T., & Heizmann, C. W. (1990). Parvalbuminimmunoreactive neurons in the rat neostriatum: A light and electron
microscope study. Brain Research, 536, 1–15.
Klopf, A. H. (1982). The hedonistic neuron: a theory of memory, learning,
and intelligence. Washington, DC: Hemisphere.
Kobatake, E., & Tanaka, K. (1994). Neuronal selectivities to complex
object features in the ventral visual pathway of the macaque cerebral
cortex. Journal of Neurophysiology, 71, 856–867.
Komatsu, H., & Ideura, Y. (1993). Relationships between color, shape, and
pattern selectivities of neurons in the inferior temporal cortex of the
monkey. Journal of Neurophysiology, 70, 677 –694.
Koos, T., & Tepper, J. M. (1999). Inhibitory control of neostriatal
projection neurons by GABAergic interneurons. Nature Neuroscience,
2, 467 –472.
Law-Tho, D., Desce, J. M., & Crepel, F. (1995). Dopamine favours the
emergence of long-term depression versus long-term potentiation in
slices of rat prefrontal cortex. Neuroscience Letters, 188, 125–128.
Lee, C., Rohrer, W. H., & Sparks, D. L. (1988). Population coding of
saccadic eye movements by neurons in the superior colliculus. Nature,
332, 357–360.
Levesque, M., Charara, A., Gagnon, S., Parent, A., & Deschenes, M. (1996).
Corticostriatal projections from layer V cells in rat are collaterals of longrange corticofugal axons. Brain Research, 709, 311–315.
Levy, R., Hazrati, L.-N., Herrero, M.-T., Vila, M., Hassani, O.-K.,
Mouroux, M., Ruberg, M., Asensi, H., Agid, Y., Feger, J., Obeso, J.,
Parent, A., & Hirsch, E. C. (1997). Re-evaluation of the functional
anatomy of the basal ganglia in normal and parkinsonian states.
Neuroscience, 76, 335–343.
Ljungberg, T., Apicella, P., & Schultz, W. (1992). Responses of monkey
dopamine neurons during learning of behavioral reactions. Journal of
Neurophysiology, 67, 145– 163.
Lo, F.-S., & Sherman, S. M. (1994). Feedback inhibition in the cat’s lateral
geniculate nucleus. Experimental Brain Research, 100, 365–368.
Logan, G. D., & Cowan, W. B. (1984). On the ability to inhibit thought and
action: A theory of an act of control. Psychologial Review, 91, 295–327.
Lynch, J. C., Hoover, J. E., & Strick, P. L. (1994). Input to the primate
frontal eye field from the substantia nigra, superior colliculus, and
dentate nucleus demonstrated by transneuronal transport. Experimental
Brain Research, 100, 181–186.
Marin, O., Sweets, W. J., & Gonzalez, A. (1998). Evolution of the basal
ganglia in tetrapods: A new perspective based on recent studies in
amphibians. Trends in Neurosciences, 21, 487–494.
Martin, K. E., Phillips, J. G., & Iansek, R. (1994). Inaccuracy and instability
of sequential movements in Parkinson’s disease. Experimental Brain
Research, 102, 131–140.
Martres, M. P., Bouthenet, M. L., Sales, N., Sokoloff, P., & Schwartz, J. C.
(1985). Widespread distribution of brain dopamine receptors evidenced
with [125I] iodosulpride, a highly selective ligand. Science, 228, 752–755.
Mayer, M. L., Westbrook, G. L., & Guthrie, P. B. (1984). Voltagedependent block by Mg2 þ of NMDA responses in spinal cord
neurones. Nature, 309, 261–263.
McFarland, N. R., & Haber, S. N. (2000). Convergent inputs from thalamic
motor nuclei and frontal cortical areas to the dorsal striatum in the
primate. Journal of Neuroscience, 20, 3798–3813.
Middleton, F. A., & Strick, P. L. (2000). Basal ganglia and cerebellar
loops: motor and cognitive circuits. Brain Research Reviews, 31,
236 –250.
Mink, J. W. (1996). The basal ganglia: focused selection and inhibition of
competing motor programs. Progress in Neurobiology, 50, 381– 425.
Mink, J. W., & Thach, W. T. (1993). Basal ganglia intrinsic circuits and
their role in behavior. Current Opinion in Neurobiology, 3, 950– 957.
Mohler, C. W., Goldberg, M. E., & Wurtz, R. H. (1973). Visual receptive
fields of frontal eye field neurons. Brain Research, 61, 385–389.
Munoz, D. P., & Wurtz, R. H. (1993). Fixation cells in monkey superior
colliculus I. Characteristics of cell discharge. Journal of Neurophysiology, 70, 559–575.
Munoz, D. P., & Wurtz, R. H. (1995). Saccade-related activity in monkey
superior colliculus I. Characteristics of burst and buildup cells. Journal
of Neurophysiology, 73, 2313– 2333.
Mushiake, H., Inase, M., & Tanji, J. (1991). Neuronal activity in the
primate premotor, supplementary, and precentral motor cortex during
visually guided and internally determined sequential movements.
Journal of Neurophysiology, 66, 705–718.
Nambu, A., Tokuno, H., & Takada, M. (2002). Functional significance of
the the cortico-subthalamo-pallidal hyperdirect pathway. Neuroscience
Research, 43, 111–117.
Oorschot, D. E. (1996). Total number of neurones in the neostriatal,
pallidal, subthalamic, and substantia nigral nuclei of the rat basal
ganglia: a stereological study using the cavalieri and optical dissector
methods. Journal of Comparative Neurology, 366, 580–599.
Otani, S., Auclair, N., Desce, J. M., Roisin, M. P., & Crepel, F. (1999).
Dopamine receptors and groups I and II mGluRs cooperate for longterm depression induction in rat prefrontal cortex through converging
postsynaptic activation of MAP kinases. Journal of Neuroscience, 19,
Ottes, F. P., Van Gisbergen, J. A., & Eggermont, J. J. (1984). Metrics of
saccade responses to visual double stimuli: Two different modes. Vision
Research, 24, 1169–1179.
Parent, A., & Hazrati, L. N. (1995). Functional anatomy of the basal
ganglia. II. The place of the subthalamic nucleus and external pallidum
in basal ganglia circuitry. Brain Research Reviews, 20, 128–154.
Parthasarathy, H. B., Schall, J. D., & Graybiel, A. M. (1992). Distributed
but convergent ordering of corticostriatal projections: analysis of the
frontal eye field and the supplementary eye field in the macaque
monkey. Journal of Neuroscience, 12, 4468–4488.
Passingham, R. (1993). The frontal lobes and voluntary action. Oxford:
Oxford University Press.
Pellizzer, G., & Hedges, J. H. (2003). Motor planning: effect of directional
uncertainty with discrete spatial cues. Experimental Brain Research,
150, 276–289.
Pierrot-Deseilligny, C., Israel, I., Berthoz, A., Rivaud, S., & Gaymard, B.
(1993). Role of the different frontal lobe areas in the control of the
horizontal component of memory-guided saccades in man. Experimental Brain Research, 95, 166–171.
Platt, M. L., & Glimcher, P. W. (1999). Neural correlates of decision
variables in parietal cortex. Nature, 400, 233–238.
Redgrave, P., Prescott, T. J., & Gurney, K. (1999). The basal ganglia: A
vertebrate solution to the selection problem? Neuroscience, 89,
Reynolds, J. N., Hyland, B. I., & Wickens, J. R. (2001). A cellular
mechanism of reward-related learning. Nature, 413, 67–70.
Royce, G. J., & Bromley, S. (1984). Florescent double labeling studies of
thalamostriatal and corticostriatal neurons. In J. S. McKenzie,
R. E. Kemm, & L. N. Wilcock (Eds.), The basal ganglia (pp.
131 –146). New York: Plenum Press.
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Sawaguchi, T. (2001). Laminar and topographic distributions of prefrontal
cortical neurons related to an oculomotor delayed response task in
monkeys. Society for Neuroscience Abstracts, 27, 85220.
Schall, J. D. (1991). Neuronal activity related to visually guided saccades in
the frontal eye fields of rhesus monkeys: Comparison with supplementary eye fields. Journal of Neurophysiology, 66, 559 –579.
Schall, J. D., & Bichot, N. P. (1998). Neural correlates of visual and motor
decision processes. Current Opinion in Neurobiology, 8, 211– 217.
Schall, J. D., Hanes, D. P., Thompson, K. G., & King, D. J. (1995b).
Saccade target selection in frontal eye field of macaque. I. Visual and
premovement activation. Journal of Neuroscience, 15, 6905–6918.
Schall, J. D., Morel, A., King, D. J., & Bullier, J. (1995a). Topography of
visual cortex connections with frontal eye field in macaque:
convergence and segregation of processing streams. Journal of
Neuroscience, 15, 4464–4487.
Schiller, P. H. (1998). The neural control of visually-guided eye
movements. In J. E. Richards (Ed.), Cognitive neuroscience of
attention. Mahwah, NJ: Lawrence Erlbaum Assoc.
Schiller, P. H., True, S. D., & Conway, J. L. (1980). Deficits in eye
movements following frontal eye field and superior colliculus ablations.
Journal of Neurophysiology, 44, 1175–1189.
Schlag, J., & Schlag-Rey, M. (1987). Evidence for a supplementary eye
field. Journal of Neurophysiology, 57, 179 –200.
Schlag-Rey, M., Amador, N., Sanchez, H., & Schlag, J. (1997).
Antisaccade performance predicted by neuronal activity in the
supplementary eye field. Nature, 390, 398 –401.
Schlag-Rey, M., & Schlag, J. (1984). Visuomotor functions of central
thalamus in monkey. II. Unit activity related to visual events, targeting,
and fixation. Journal of Neurophysiology, 51, 1175– 1195.
Schultz, W. (1998). Predictive reward signals of dopamine neurons.
Journal of Neurophysiology, 80, 1– 27.
Segraves, M. A., & Goldberg, M. E. (1987). Functional properties of
corticotectal neurons in the monkey’s frontal eye field. Journal of
Neurophysiology, 58, 1387–1419.
Shink, E., Bevan, M. D., Bolam, J. P., & Smith, Y. (1996). The subthalamic
nucleus and the external pallidum: two tightly interconnected structures
that control the output of the basal ganglia in the monkey.
Neuroscience, 73, 335–357.
Sigala, N., & Logothetis, N. K. (2002). Visual categorization shapes feature
selectivity in the primate temporal cortex. Nature, 415, 318 –320.
Skinner, R. D., & Garcia-Rill, E. (1990). Brainstem modulation of rhythmic
functions and behaviors. In W. R. Klemm, & R. P. Vertes (Eds.),
Brainstem mechanisms of behavior (pp. 465 –496). New York: Wiley.
Smith, Y., & Bolam, J. P. (1990). The output neurones and the
dopaminergic neurones of the substantia nigra receive a GABAcontaining input from the globus pallidus in the rat. Journal of
Comparative Neurology, 296, 47–64.
Sommer, M. A., & Tehovnik, E. J. (1999). Reversible inactivation of
macaque dorsomedial frontal cortex: effects on saccades and fixations.
Experimental Brain Research, 124, 429 –446.
Sommer, M. A., & Wurtz, R. H. (1998). Frontal eye field neurons
orthodromically activated from the superior colliculus. Journal of
Neurophysiology, 80, 3331–3333.
Sommer, M. A., & Wurtz, R. H. (2000). Composition and topographic
organization of signals sent from the frontal eye field to the superior
colliculus. Journal of Neurophysiology, 83, 1979–2001.
Steele, G. E., & Weller, R. E. (1993). Subcortical connections of
subdivisions of inferior temporal cortex in squirrel monkeys. Visual
Neuroscience, 10, 563–583.
Steiner, H., & Gerfen, C. R. (1998). Role of dynorphin and enkephalin in
the regulation of striatal output pathways and behavior. Experimental
Brain Research, 123, 60 –76.
Strick, P. L., Dum, R. P., & Picard, N. (1995). Macro-organization of the
circuits connecting the basal ganglia with the cortical motor areas.
In J. Houk, J. Davis, & D. Beiser (Eds.), Models of information
processing in the basal ganglia (pp. 117– 130). Cambridge: MIT Press.
Sung, K. W., Choi, S., & Lovinger, D. M. (2001). Activation of group I
mGluRs is necessary for induction of long-term depression at striatal
synapses. Journal of Neurophysiology, 86, 2405–2412.
Suri, R. E., Albani, C., & Glattfelder, A. H. (1997). A dynamic model of
motor basal ganglia functions. Biological Cybernetics, 76, 451–458.
Suri, R. E., Bargas, J., & Arbib, M. A. (2001). Modeling functions of striatal
dopamine modulation in learning and planning. Neuroscience, 103,
65– 85.
Sutton, R. S., & Barto, A. G. (1981). Toward a modern theory of adaptive
networks: expectation and prediction. Psychological Review, 88,
135– 170.
Sweeney, J. A., Mintun, M. A., Kwee, S., Wiseman, M. B., Brown, D. L.,
Rosenberg, D. R., & Carl, J. R. (1996). Positron emission tomography
study of voluntary saccadic eye movements and spatial working
memory. Journal of Neurophysiology, 75, 454–468.
Takikawa, Y., Kawagoe, R., Itoh, H., Nakahara, H., & Hikosaka, O. (2002).
Modulation of saccadic eye movements by predicted reward outcome.
Experimental Brain Research, 142, 284–291.
Tanaka, K., Saito, H., Fukada, Y., & Moriya, M. (1991). Coding visual
images of objects in the inferotemporal cortex of the macaque monkey.
Journal of Neurophysiology, 66, 170–189.
Taylor, J. G., & Taylor, N. R. (2000). Analysis of recurrent cortico-basal
ganglia-thalamic loops for working memory. Biological Cybernetics,
82, 415–432.
Thompson, K. G., Bichot, N. P., & Schall, J. D. (1997). Dissociation of
visual discrimination from saccade programming in macaque frontal
eye field. Journal of Neurophysiology, 77, 1046–1050.
Turner, R. S., & Anderson, M. E. (1997). Pallidal discharge related to the
kinematics of reaching movements in two dimensions. Journal of
Neurophysiology, 77, 1051–1074.
Turner, R. S., & DeLong, M. R. (2000). Corticostriatal activity in
primary motor cortex of the macaque. Journal of Neuroscience, 20,
Turner, R. S., Grafton, S. T., Votaw, J. R., Delong, M. R., & Hoffman, J. M.
(1998). Motor subcircuits mediating the control of movement velocity:
a PET study. Journal of Neurophysiology, 80, 2162–2176.
Vezina, P., & Kim, J. H. (1999). Metabotropic glutamate receptors and the
generation of locomotor activity: interactions with midbrain dopamine.
Neuroscience and Biobehavioral Reviews, 23, 577–589.
Weiss, P., Stelmach, G. E., & Hefter, H. (1997). Programming of a
movement sequence in Parkinson’s disease. Brain, 120, 91– 102.
Wichmann, T., Bergman, H., & DeLong, M. R. (1994). The primate
subthalamic nucleus. Journal of Neurophysiology, 72, 494–506.
Wichmann, T., & DeLong, M. (1996). Functional and pathophysiological
models of the basal ganglia. Current Opinion in Neurobiology, 6,
751– 758.
Wichmann T., & DeLong, M., (2001). Basal ganglia circuits in movement
and movement disorders. In K. Kultas-Llinsky and I. A. Llinsky (Eds.).
Basal ganglia and thalamus in health and movement disorders (pp. 1125). Kluwer Academic.
Wickens, J. (1997). Basal ganglia: Structure and computations. Network:
Computation in Neural Systems, 8, R77–R109.
Wickens, J., Begg, A., & Arbuthnott, G. (1996). Dopamine reverses
the depression of rat corticostriatal synapses which normally
follows high-frequency stimulation of cortex in vitro. Neuroscience,
70, 1–5.
Wilson, C. (1995a). The contribution of cortical neurons to the firing
pattern of striatal spiny neurons. In J. Houk, J. Davis, & D. Beiser
(Eds.), Models of information processing in the basal ganglia (pp.
29– 50). Cambridge, MA: MIT Press.
Wilson, C. (1995b). Corticostriatal neurons of the medial agranular cortex
of rats. In M. Kimura, & A. Graybiel (Eds.), Functions of the corticobasal ganglia loop (pp. 50–72). Tokyo: Springer-Verlag.
Wilson, C. J., Kita, H., & Kawaguchi, Y. (1989). GABAergic interneurons,
rather than spiny cell axon collaterals, are responsible for the IPSP
responses to afferent stimulation in neostriatal spiny neurons. Society
for Neuroscience Abstracts, 15, 907.
J.W. Brown et al. / Neural Networks 17 (2004) 471–510
Wise, S. P., & Murray, E. A. (2000). Arbitrary associations between
antecedents and actions. Trends in Neurosciences, 23, 271 –276.
Wurtz, R. H., Sommer, M. A., Pare, M., & Ferraina, S. (2001).
Signal transformations from cerebral cortex to superior colliculus
for the generation of saccades. Vision Research, 41, 3399–3412.
Yeterian, E. H., & Hoesen, G. W. V. (1978). Cortico-striate projections in
the rhesus monkey: The organization of certain cortico-caudate
connections. Brain Research, 139, 43 –63.
Yeterian, E. H., & Pandya, D. N. (1994). Laminar origin of striatal and
thalamic projections of the prefrontal cortex in rhesus monkeys.
Experimental Brain Research, 99, 383 –398.
Young, A. B., & Penney, J. B. (2001). What we know and what we have left
to learn. In K. Kultas-Ilinsky, & I. A. Ilinsky (Eds.), Basal ganglia and
thalamus in health and movement disorders (pp. 3–10). Dordrecht:
Academic Press.
Zheng, T., & Wilson, C. J. (2002). Corticostriatal combinatorics: The
implications of cortico-striatal axonal arborizations. Journal of
Neurophysiology, 87, 1007–1017.
Zin-Ka-Ieu, S., Roger, M., & Arnault, P. (1998). Direct contacts between
fibers from the ventrolateral thalamic nucleus and frontal cortical
neurons projecting to the striatum: a light-microscopy study in the rat.
Anatomy and Embryology (Berlin), 197, 77–87.
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Related manuals

Download PDF