Stan Van Pelt and W. Pieter Medendorp

Stan Van Pelt and W. Pieter Medendorp
J Neurophysiol 99:2281-2290, 2008. First published Mar 19, 2008; doi:10.1152/jn.01281.2007
You might find this additional information useful...
This article cites 55 articles, 32 of which you can access free at:
http://jn.physiology.org/cgi/content/full/99/5/2281#BIBL
Updated information and services including high-resolution figures, can be found at:
http://jn.physiology.org/cgi/content/full/99/5/2281
Additional material and information about Journal of Neurophysiology can be found at:
http://www.the-aps.org/publications/jn
This information is current as of August 11, 2008 .
Downloaded from jn.physiology.org on August 11, 2008
Journal of Neurophysiology publishes original articles on the function of the nervous system. It is published 12 times a year
(monthly) by the American Physiological Society, 9650 Rockville Pike, Bethesda MD 20814-3991. Copyright © 2005 by the
American Physiological Society. ISSN: 0022-3077, ESSN: 1522-1598. Visit our website at http://www.the-aps.org/.
J Neurophysiol 99: 2281–2290, 2008.
First published March 19, 2008; doi:10.1152/jn.01281.2007.
Updating Target Distance Across Eye Movements in Depth
Stan Van Pelt1 and W. Pieter Medendorp1,2
1
Nijmegen Institute for Cognition and Information, 2FC Donders Centre for Cognitive Neuroimaging, Radboud University Nijmegen,
Nijmegen, The Netherlands
Submitted 23 November 2007; accepted in final form 16 March 2008
INTRODUCTION
Maintaining spatial constancy across self-generated movements is crucial for veridical perception of the world and for
accurate control of goal-directed actions. Over the past few
decades, the quality of spatial constancy has been investigated
systematically across various types of self-motion, including
eye, head, and body movements. As a result, it is now well
established that spatial constancy is preserved across intervening saccadic (Hallet and Lightstone 1976; Sparks and Mays
1983) and smooth pursuit eye movements (Baker et al. 2003;
Schlag et al. 1990). Also a reorientation of the head or
displacement of the body in space does not compromise spatial
stability to a great extent (Israel et al. 1993; Li and Angelaki
2005; Medendorp et al. 1999, 2002, 2003b; Mergner et al.
2001; Van Pelt and Medendorp 2007; Van Pelt et al. 2005).
From a mechanistic perspective, there has been considerable
debate over how the brain solves the spatial constancy problem. In the absence of allocentric cues, it seems that an
egocentric, gaze-centered reference frame dominates in the
mechanisms of spatial stability for simple saccade or reaching
Address for reprint requests and other correspondence: W. P. Medendorp,
Nijmegen Institute for Cognition and Information, Radboud University Nijmegen, P.O. Box 9104, NL-6500 HE, Nijmegen, The Netherlands (E-mail:
[email protected]).
www.jn.org
tasks (Henriques et al. 1998; Klier et al. 2005; Medendorp and
Crawford 2002; Van Pelt and Medendorp 2007). In support,
cells in monkey extrastriate visual areas (Nakamura and Colby
2002), posterior parietal cortex (Batista et al. 1999; Colby and
Goldberg 1999; Duhamel et al. 1992), frontal cortex (Goldberg
and Bruce 1990), and superior colliculus (Walker et al. 1995),
as well as in the human posterior parietal cortex (Medendorp
et al. 2003a; Merriam et al. 2003, 2007) have been shown to
update the gaze-centered coordinates of remembered stimuli
to maintain an accurate representation of visual space across
saccades.
Notwithstanding these convincing observations and clear
insights, it should be emphasized that nearly all these studies
were limited by only examining the directional aspect of spatial
constancy. For many spatially guided actions, however, directional constancy is not the only spatial requirement; the constancy of target depth (or distance) is another essential component that should be mediated by the signals and mechanisms
for spatial stability.
Because it is generally assumed that target depth and direction are processed in functionally distinct visuomotor channels
(Cumming and DeAngelis 2001; DeAngelis 2000; Flanders
et al. 1992; Vindras et al. 2005), the mechanisms to preserve
their constancy may also operate independently, at least to some
extent. To date, only few studies have explicitly assessed the
constancy of target depth during self-motion (Krommenhoek and
Van Gisbergen 1994; Li and Angelaki 2005; Medendorp et al.
1999, 2003b; Philbeck and Loomis 1997). Krommenhoek and
Van Gisbergen (1994) showed that subjects can look at a
remembered position of a target in depth after a vergence eye
movement. Li and Angelaki (2005) reported that non-human
primates can keep track of changes in the distance of nearby
objects when their body moved toward or away from them.
Despite these quantitative observations, the computational
mechanisms underlying depth constancy have not been
addressed. The objective of the present study is to fill this
lacuna by testing between two models for depth coding in
the visuomotor system.
While a variety of cues to depth can be used by the visual
system, binocular disparity dominates in the creation of a
cohesive, three-dimensional depth perception (Howard and
Rogers 1995; Julesz 1971; Wei et al. 2003). Binocular disparity
is caused by the slight difference in viewpoint of the two eyes
due to their differential location in the head. Objects at different distances from the eyes’ fixation distance project onto
different positions on each retina and thus cause different
horizontal binocular disparities. Likewise, a single object at a
The costs of publication of this article were defrayed in part by the payment
of page charges. The article must therefore be hereby marked “advertisement”
in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
0022-3077/08 $8.00 Copyright © 2008 The American Physiological Society
2281
Downloaded from jn.physiology.org on August 11, 2008
Van Pelt S, Medendorp WP. Updating target distance across eye
movements in depth. J Neurophysiol 99: 2281–2290, 2008. First
published March 19, 2008; doi:10.1152/jn.01281.2007. We tested
between two coding mechanisms that the brain may use to retain
distance information about a target for a reaching movement across
vergence eye movements. If the brain was to encode a retinal disparity
representation (retinal model), i.e., target depth relative to the plane of
fixation, each vergence eye movement would require an active update
of this representation to preserve depth constancy. Alternatively, if the
brain was to store an egocentric distance representation of the target
by integrating retinal disparity and vergence signals at the moment of
target presentation, this representation should remain stable across
subsequent vergence shifts (nonretinal model). We tested between
these schemes by measuring errors of human reaching movements
(n ⫽ 14 subjects) to remembered targets, briefly presented before a
vergence eye movement. For comparison, we also tested their directional accuracy across version eye movements. With intervening
vergence shifts, the memory-guided reaches showed an error pattern
that was based on the new eye position and on the depth of the
remembered target relative to that position. This suggests that target
depth is recomputed after the gaze shift, supporting the retinal model.
Our results also confirm earlier literature showing retinal updating of
target direction. Furthermore, regression analyses revealed updating
gains close to one for both target depth and direction, suggesting that
the errors arise after the updating stage during the subsequent reference frame transformations that are involved in reaching.
2282
S. VAN PELT AND W. P. MEDENDORP
A
B
DEPTH
STATIC
FP Far
FP Left
FP Near
FP Right
FIG. 1. Testing between nonretinal and retinal models for
memory encoding of spatial depth and direction. A and B: static
condition: it is supposed that the reach toward a space-fixed
target will be erred depending on gaze fixation position in depth
(A) and direction (B). The exact relationship is not of importance for the test. C and D: dynamic condition: a gaze shift
intervenes between target presentation and reaching. The nonretinal model predicts no effect of the gaze shift on reaching.
The retinal scheme requires target updating relative to the new
gaze position, predicting reach errors as in the static situation
with the eyes at the same final position.
T
C
FP Far►Near
Nonretinal
Retinal
D
FP Left►Right
Nonretinal
Retinal
ΔEye
ΔEye
DYNAMIC
DIRECTION
J Neurophysiol • VOL
depicts the situation for the depth dimension when subjects
changed gaze from far to near fixation after initial target
perception. Reaching in depth, as in the static case without an
intervening vergence eye movement (Fig. 1A, static FP far
condition), would argue in favor of the use of a nonretinal
depth representation. However, if the intervening eye movement leads to a depth error like that observed when the same
target was viewed from the final eye position (Fig. 1A, static FP
near condition), this would provide evidence for the use of an
updated eye-centered binocular disparity representation. Following its original design, the test can likewise discriminate
between a retinal and a nonretinal representation of target
direction across saccadic eye movements (as shown by the
panels in Fig. 1D).
Our results suggest that the brain codes dynamic disparity
and direction representations to store target locations for reaching across eye movements in depth and direction. Regression
analyses revealed that these representations are modulated by
using both eye position and eye displacement signals consistent
with recent observations in monkey neurophysiology (Genovesio
et al. 2007).
METHODS
Subjects
Fourteen human subjects (4 female, 10 male; mean age: 26 ⫾ 4 yr)
signed informed consent to participate in this study. All were free of
any known sensory, perceptual, or motor disorders. Twelve participants were right-handed; two were left-handed; reaching movements
were made using the preferred arm. Two subjects (the authors) were
aware of the purpose of the experiments, whereas the others were
naı̈ve.
Experimental setup
Subjects were seated in a completely darkened room with their
torso securely strapped into a custom-made chair by means of two
safety belts across both the torso and pelvis to minimize body
movement. Their head was mechanically stabilized using a chin rest
and a helmet, which was fixed to the chair by means of a frame that
was adjustable in height. This ensured that only the preferred arm and
the eyes could move while the rest of the body remained stationary.
99 • MAY 2008 •
www.jn.org
Downloaded from jn.physiology.org on August 11, 2008
fixed position from the eyes will have different horizontal
disparities for different viewing distances.
In this study, we investigated how the brain codes the
distance of a remembered space-fixed target during intervening
changes of the binocular fixation point (i.e., vergence eye
movements). We reasoned that if the brain was to encode a
binocular disparity representation, i.e., target depth relative to
the eyes’ fixation point (Shadmehr and Wise 2005), each
disjunctive change of gaze will require an active update of this
representation to maintain spatial constancy. Alternatively, if
the brain was to store a nonretinal depth representation of the
target by integrating binocular disparity and vergence signals at
the moment of target presentation (Genovesio and Ferraina
2004; Genovesio et al. 2007), this representation should remain
stable for subsequent vergence eye movements.
To test between these hypotheses, we employed a memoryguided reach paradigm adopted from Henriques et al. (1998),
who originally developed it to examine the computations for
directional spatial constancy. We expanded this test by examining actual versus predicted localization errors in depth when
vergence eye movements intervene between viewing a target
and reaching toward its remembered location as will be further
outlined in Fig. 1.
The assumption behind our test was that subjects make
systematic distance errors in their reach toward memorized
targets, depending on their fixation depth (static reaching, Fig.
1A)—as they have shown to make directional errors depending
on their gaze direction (Fig. 1B) (Henriques et al. 1998). In the
latter case, the phenomenon has been termed the “retinal
exaggeration effect” because subjects tend to overshoot the
target relative to current gaze direction although individual
subjects show considerable variations in this pattern (Bock
1986; Henriques and Crawford 2000). It is not known if a
similar overshoot effect occurs for depth; however, as long as
distance errors depend on fixation depth, even if only in a
complex and idiosyncratic manner, this relationship can be
exploited to distinguish between retinal and nonretinal target
representations.
The critical part of the test is based on the errors that occur
when subjects reach after an intervening eye movement toward
the location of a target that was viewed only before this eye
movement (dynamic situation, Fig. 1, C and D). Figure 1C
DEPTH CODING FOR REACHING MOVEMENTS
4.1 – 6.5
A
2.5 – 4
2 – 2.5
1.5 – 2
FP2
0 - 1.5
T
FP1
(s)
Time
B
Stimulus
Configuration
Version
Change
Vergence
Change
Combined
Change
8°
10°
0°
-10°
°
13 8°
1
10cm
FIG. 2. Experimental paradigm. A: sequence of stimuli and the subject’s
instructions. A trial started with the illumination of a red fixation light (FP1).
Then, after a delay of 1.5 s, a green target (T) was cued for 0.5 s. After a further
0.5 s, the subject had to change fixation to fixation light 2 (FP2) in dynamic
trials. In static trials, fixation was to be kept at FP1, thus FP2 ⫽ FP1. Next,
1.5 s later, FP2 was extinguished, and an auditory cue instructed the subject to
reach toward the remembered location of the target, while keeping fixation at
the remembered location of FP2. Open circles, possible stimulus locations;
filled circles, exemplar stimuli presented. B: potential locations of the stimuli,
which served either as (initial or final) fixation point or memory target, or both,
were on the intersections of 3 isoversion (⫺10, 0, and 10°) and 3 isovergence
(8, 13, and 18°) lines. In dynamic trials, gaze displacements could consist of a
pure version movement (2nd panel), a pure vergence change (3rd panel), or a
combination of both (rightmost panel).
J Neurophysiol • VOL
Optotrak data were sampled at 125 Hz with an accuracy of ⬍0.2 mm
and saved on a PC for off-line analysis.
We recorded the subjects’ binocular eye movements using an
Eyelink II eyetracker (SR Research) mounted to the chair-fixed
helmet. This system tracks the pupils’ positions using infrared light
reflection at a sampling rate of 250 Hz. Before the experiment began,
eye movements were calibrated by fixating the stimulus LEDs three
times each, in complete darkness. This resulted in a calibration
accuracy ⬍0.5°. Calibration was checked off-line, to allow for drift
correction due to headband slippage or other factors. Because the head
and body stayed fixed during the experiment, the orientation of the
eyes within the head, as measured by the tracker, was equivalent to the
orientation of the eyes in space (gaze). Rightward rotations were taken
as positive.
Two PCs in a master-slave arrangement controlled the experiment.
The master PC contained hardware for data acquisition of the Optotrak measurements and visual stimulus control. The slave PC was
equipped with hardware and software from the Eyelink system.
Experimental paradigm
The main focus of this study is to reveal the reference frame
employed by the brain to maintain spatial constancy for depth. To
allow for comparison with previous studies (Beurze et al. 2006;
Henriques et al. 1998; Medendorp and Crawford 2002), we employed
a paradigm that also tested the mechanisms for directional constancy.
Figure 2A illustrates the paradigm. A trial started with the onset of
a red fixation LED, which we refer to as FP1 (fixation point 1), to be
fixated for its entire illumination duration of 2.5 s. FP1 could be any
of the nine stimulus locations on the stimulus array (Fig. 2B, leftmost
panel). At 1.5 s after the onset of FP1, a target for memory (T, a green
LED) was flashed for 0.5 s while the subject kept gaze fixed at FP1.
Thus T was on the fovea when presented at the same location of FP1
but on the peripheral retina for any of the eight other possible
locations. Next, 0.5 s after the flash, a time interval of 1.5 s followed
during which the subject either changed gaze fixation to a second
illuminated fixation light (FP2— dynamic paradigm) or maintained
fixation of the first fixation point when FP2 ⫽ FP1 (static paradigm).
Subsequently, at FP2 offset (4.0 s after trial onset), the stimulus array
was retracted, followed 100 ms later by an auditory signal that cued
the subject to reach to T, while keeping gaze fixed at (the remembered
location of) FP2. The subject had to hold the reaching position until
the end of a 2.4-s interval, indicated by a second auditory signal. Then
the next trial started, with FP1 at a different location than the location
of T in the preceding trial, to avoid any visual feedback about performance in the previous trial. Between trials, subjects had their reaching
arms resting unencumbered on their lap with the hand close to their
knees. FP1, T, and FP2 were pseudorandomly selected from the
stimulus array, such that all combinations of FP1, T, and FP2 were
tested once. This yielded a total of 729 unique trials: 81 trials were
pure static trials (FP1 ⫽ FP2) and the other trials were dynamic trials.
Of the dynamic trials, 162 trials had a pure conjugate change in eye
position (version eye movement), whereas vergence (disjunctive part)
remained constant (Fig. 2B, 2nd panel); 162 trials had a vergence
change but constant version (ignoring the small vertical version eye
movements due to the vertical offset in the positioning of near and far
LEDs; 3rd panel ); and 324 trials had a combined vergence-version
change (rightmost panel ).
The total experiment was divided into three sessions, each of which
lasted for ⬃60 min each and were tested on different days. In each
session, subjects performed blocks of 15 or 16 consecutive trials
between which a brief rest was provided with the room lights on to
avoid dark adaptation. During the experiments, subjects never received feedback about their performance. Before the actual experiments, subjects practiced a few blocks to become familiar with the
task.
99 • MAY 2008 •
www.jn.org
Downloaded from jn.physiology.org on August 11, 2008
The stimulus array (see Fig. 2B, left) consisted of nine lightemitting diodes (LEDs), each 3 mm in diameter, and each could be
flashed in two different colors, either as a green or a red light
(luminance ⬍20 mcd/m2). The LEDs had fixed positions on a frame,
which could be moved by a robotic arm. Stimuli were presented in
front of the subject, in a horizontal plane, slightly below the eyes, at
the intersections of three imaginary horopters (equal vergence lines: 8,
13, and 18° vergence, i.e., 46.5, 28.5, and 20.5 cm from the subjects
eyes) and three equidirection lines (⫺10, 0, and ⫹10° version), based
on an average interocular distance of 6.5 cm. The robotic arm was
equipped with stepping motors (type Animatics SmartMotors, Servo
Systems) and could rapidly move the stimulus array to various
positions within the workspace, bringing it within 200 ms out of touch
during the reaching task performed by the subject (see following text)
(see also Van Pelt and Medendorp 2007). During the experiments, the
total movement time of the robot was always 2.3 s. Also between
trials, when the room lights were on, the stimulus array was close to
the ceiling of the experimental room. This way only the frame’s rear
side could be viewed, which this gave no information about the spatial
configuration of the stimuli.
Prior to the experiments, we measured the location of the eyes in
space and the locations of the space-fixed stimulus LEDs using an
Optotrak Certus system (Northern Digital). With this information, we
were able to compute the direction and distance of the stimulus LEDs
with respect to the subject’s eyes. During the experiment, the Optotrak
continuously recorded the location of the tip of the index finger. We
ensured that the fingertip was at least always visible during the last
part of the reaching movement (see Van Pelt and Medendorp 2007).
2283
S. VAN PELT AND W. P. MEDENDORP
Data analysis
DEPTH
RESULTS
The experiments were designed to test between retinal and
nonretinal models of the coding of target depth and direction.
The basic premise for this test is a difference in accuracy of
reaching movements toward remembered space-fixed but nonfoveally viewed targets for different eye fixation positions
(Beurze et al. 2006; Henriques et al. 1998). The two models
make clearly different predictions about the reach errors that
would arise when a gaze displacement intervenes between
seeing the target and reaching to its remembered location
(dynamic condition). The nonretinal model predicts an error
similar to that observed in the static condition without the
intervening gaze displacement, whereas the retinal model predicts an error similar to that observed for a target viewed from
the same final fixation position (see Fig. 1).
Task performance
Figure 3 illustrates task performance of a typical subject,
demonstrating static and dynamic trials testing either for depth
constancy (left) or for direction constancy (right). The left
panels show binocular vergence (gray traces) and measured
fingertip depth (black traces) over the time course of eight trials
J Neurophysiol • VOL
B
Far Fixation
Left Fixation
20
20
15
10
5
0
10
0
-10
T
R
FP1
T
20
15
10
5
0
Right Fixation
T
C
DYNAMIC
R
FP1
R
FP1
-20
Near Fixation
Vergence (°)
STATIC
A
20
10
0
-10
T
D
Far ►Near
R
FP1
-20
Left ►Right
20
20
15
10
5
0
10
0
-10
T
FP1
0
R
FP2
2
4
T
FP1
-20
0
6
R
FP2
2
4
6
Time (s)
Eye
Finger
Ideal Eye
Target
FIG. 3. Typical performance of 1 subject. Eye position (version, vergence,
in gray), target (dotted), fingertip position (in black), expressed in binocular
depth and direction coordinates (in deg), plotted as function of time. Dashed
trace, geometrically ideal eye position. Fingertip traces are only shown for the
last 1.5 s of the reach period. Left panels: investigation of depth coding of a
target at 13° vergence angle, with 2 static conditions (far vs. near fixation, A),
and the corresponding dynamic condition (far to near, C). Right panels:
investigation of directional coding of a craniotopically central target with 2
static conditions (left vs. right fixation, B), and the corresponding dynamic
condition (D). Thin boxes, time intervals of the different trial stages [target
presentation (T), FP1 and FP2 periods, reach interval (R)].
with a target flashed at the middle depth (13° vergence when
foveated, dotted line, see also Fig. 2B). Following instructions,
the eyes fixated at either one of the far targets (Fig. 3A, top,
requiring a smaller vergence angle) or at one of the near targets
(Fig. 3A, top, requiring a larger vergence angle) or reoriented
from far to near fixation after stimulus presentation (Fig. 3C).
In all conditions, final eye fixation, which was to be maintained
during the reach, showed a small decline in vergence after the
offset of the fixation point, at the go cue for the reach, but note
that reach responses were performed in complete darkness. In
the static trials, reaches (black) showed small overshoots
(smaller response angle than required) depending on the eyes’
fixation depth, with errors of about ⫺2.2 ⫾ 0.8° and ⫺4.5 ⫾
0.9° for far and near fixation. In the dynamic trials, in which
the target is viewed with far fixation and the reach is performed
with the eyes fixating near (Fig. 3C), the errors seem qualitatively indistinguishable from those in the static near situation
(Fig. 3A, bottom) with a mean error of ⫺4.7 ⫾ 0.7°. Thus a
change of gaze in depth affects the reaching responses to
previously seen targets, making them look like those with gaze
stationary at the same final depth.
Figure 3, B and D, shows eight typical time courses in each
of three trial types serving to illustrate how the directional
coding of a craniotopically central target depends on (changes in)
99 • MAY 2008 •
www.jn.org
Downloaded from jn.physiology.org on August 11, 2008
Data were analyzed off-line using Matlab (The Mathworks). Optotrak data were first transformed to a right-handed Cartesian coordinate system, referenced to the position of the cyclopean eye. In this
coordinate system, the positive y axis pointed leftward along the
shoulder line (from the subject’s perspective), the x axis pointed
forward and the z axis upward.
Horizontal gaze direction was computed for each eye separately;
binocular version and vergence angles were calculated from the left
(L) and right (R) gaze directions as (R ⫹ L)/2 and L ⫺ R, respectively.
Rightward rotations were taken as positive. Cartesian positions of
FP1, FP2, T, and the fingertip were also expressed in binocular
coordinates, in terms of depth and direction (in degrees) from the
cyclopean eye. This allowed for the computation of reach and target
depth relative to the plane of fixation, expressed in terms of angular
disparity (Howard and Rogers 1995). By convention, crossed disparities were taken as positive.
We discarded trials in which subjects did not maintain fixation
within a 5 ⫻ 4° (version ⫻ vergence) interval around the fixation
points or made a saccade during target presentation. For the remaining
trials, eye fixation accuracy was 2.60 ⫾ 0.83° (mean ⫾ SD). We also
excluded trials in which subjects had not correctly followed the
reaching instructions of the paradigm, i.e., when they started their
reaching movement too early or did not adopt a stable reach position
during the response intervals (fingertip velocity ⬎5 cm/s based on
Optotrak data). Overall ⬍3% of the trials was discarded on the basis
of these arm and eye movement criteria.
The endpoint of each reaching movement was selected at the time
at which the velocity of the fingertip first dropped below 5 cm/s within
the 2.4-s reaching interval, under the requirement that the arm had
correctly followed the instructions of the paradigm. An average
position was computed over an eight-sample interval (64 ms) centered
at this time point.
Analyses were performed separately for the directional and depth
dimensions. We assessed performance by quantifying reach errors in
both dimensions for each trial. Using multiple linear regression
analysis, we investigated the effects of eye displacement and eye
position on the reach errors that were observed. Statistical tests were
performed at the 0.05 level (P ⬍ 0.05).
DIRECTION
Version (°)
2284
DEPTH CODING FOR REACHING MOVEMENTS
gaze direction. Again, in all trials, the eyes act according to
instructions, showing steady fixation during target presentation
and only small saccades at the moment of reach. Reaching
behavior in the dynamic trials, in which gaze changes from 10°
leftward to 10° rightward direction (Fig. 3D), matches more
closely the observations made in static trials with gaze in the
10° rightward direction than with gaze at the same direction of
stimulus presentation (Fig. 3B). For the static trials, mean
horizontal reach error was 2.5 ⫾ 2.8 and ⫺12.1 ⫾ 3.5° in the
static left and right conditions and ⫺11.5 ⫾ 3.4°) in the
dynamic condition. In other words, the change in eye position
has had a marked effect on reaching behavior.
Reach patterns
DEPTH
FP Far
DIRECTION
FP Near
B
FP Left
FP Right
STATIC
A
portantly, reach patterns in the two static conditions seem
noticeably different, depending on the eyes’ fixation depth.
The question is which of these reach patterns is observed in
a dynamic condition in which the subject viewed the target
with far fixation, then changed gaze toward near fixation and
subsequently reached toward the remembered target location.
If the subject had stored absolute target depth relative to the
body (nonretinal model), computed at the time of seeing the
target, the intervening gaze deviation should have no systematic effect on the reach responses. Thus the nonretinal model
predicts reach errors as in the static condition with gaze in far
space. However, if the subject had stored target depth relative
to the plane of fixation, as a retinal disparity signal (retinal
model), this signal must be updated for the gaze change,
predicting an error pattern similar to that observed in the static
condition with near fixation.
Figure 4C shows the systematic reach patterns obtained
in the related dynamic trials (thick black lines), superimposed
on the predictions of either of the two models. Note that we
based this illustration on “pure” trials only, i.e., trials with a
change in vergence (far-to-near fixation) but constant version
(see METHODS), to demonstrate the effect of the vergence
change in the clearest possible fashion. It is important to realize
that using “combined” trials here (trials with a change in both
vergence and version, see METHODS) could easily obscure the
main effects in either dimension. Clearly for this subject and
when gaze was displaced from far to near (Fig. 4C), the data
seem more consistent with the predictions of the retinal model
Nonretinal
Retinal
Retinal
FP Far►Near
FP Left►Rigth
D
RMSE=2.2°
RMSE=0.8°
RMSE=12.7°
RMSE=3.1°
RMSE=14.3°
RMSE=4.7°
F
FP Right►Left
E
FP Near►Far
DYNAMIC
C
Nonretinal
RMSE=2.0°
RMSE=0.7°
J Neurophysiol • VOL
99 • MAY 2008 •
www.jn.org
FIG. 4. Reach patterns across all targets
from 1 subject (RV). A: depth coding with
reach patterns (in gray) in 2 static conditions
(far vs. near), superimposed on the target grid
(in thin black). C and E: reach pattern in the
corresponding dynamic conditions (in black),
superimposed on the patterns (in gray) from A,
that serve as predictions of the nonretinal and
retinal models. B: directional coding with
reach patterns in 2 static conditions superimposed on target grid. D and F: reach patterns in
the corresponding dynamic conditions, superimposed on the predicted patterns by the 2
models. root-mean-squared error (RMSE, in
deg), a measure of the deviation of the dynamic data to the model prediction. Each data
point represents the mean of 3 pure trials with
version as the free parameter for the depth
trials (A, C, and E) and vergence for the
direction trials (B, D, and F) dimension, respectively. Circle size, SE of each data point.
Downloaded from jn.physiology.org on August 11, 2008
To demonstrate performance in the static and dynamic
conditions more clearly, Fig. 4 shows spatial plots of reach
endpoints (filled circles) for a single subject (RV). The size of
each circle represents the corresponding confidence limit (see
legend for computation). In the static conditions with far or
near fixation (Fig. 4A), the mean reach endpoints toward the
nine targets are interconnected with gray lines, and superimposed on the spatial structure defined by the stimulus locations
(thin black lines). Perfect behavior would require that reach
responses align with the stimulus matrix. This is clearly not the
case: the subject makes substantial errors with regard to both
depth and direction of nearly all target locations. Reaching
movements of this subject undershoot the distance of some
targets, whereas they are more accurate in others. More im-
2285
S. VAN PELT AND W. P. MEDENDORP
J Neurophysiol • VOL
A
B
Depth
4
16
N=18 manipulations
Direction
N=18
3
8
2
1
2
3
RMSE Nonretinal (°)
4
0
8
16
RMSE Nonretinal (°)
FIG. 5. RMS errors in relation to the 2 models plotted vs. each other. Data
pooled across trials that have the same gaze displacement (same dynamic
situation), in 1 subject (RV, same as in Fig. 4). For both depth (A) and direction
(B), most data points fall below the diagonal, indicating that the retinal model
gives a better description of the data in this subject.
Next, before we proceed further, recall that the efficacy of
our test is based on the premise that the reach error in the static
condition depends on eye position. Figures 4 and 5 confirm this
for a single subject. To test this assumption in an analysis
across subjects, we performed a 3 ⫻ 3 repeated-measures
ANOVA on the static reach errors with eye position and
craniotopic target location as within-subject factors. This analysis revealed a significant effect of either factor, in both the
depth and direction dimension [in all tests F(2,12)⬎6.3, P ⬍
0.05], which validates the basic premise of our test.
Subsequently, under this confirmed assumption, Fig. 6 quantifies the results of all 14 subjects in two typical dynamic
conditions (far-to-near, near-to-far and left-to-right, right-toleft). Analogous to the observations made in Fig. 4, the figure
demonstrates that the retinal model fits the mean pattern of
reach errors across subjects better than the nonretinal model
(compare the corresponding RMSE values). The mean RMSE
values across all testing conditions are given in Fig. 7, A and B,
for each subject separately. Across our population of subjects,
the retinal model produced the best description for the coding
of both target distance and direction (paired t-test, P ⬍ 0.001).
Model analysis
Although the data of our subjects seem to lend support for
the retinal model, this interpretation may be flawed if reach
errors were to depend nonretinally on final eye position instead
of being caused by an updated retinal representation. To
examine this, in the following analysis, we further quantified
reaching behavior by performing a multiple linear regression to
investigate how the reach error relates to either eye displacement or final eye position. We fitted the following relationship,
separately for the depth and direction dimension
Err ⫽ a0 ⫹ a1 䡠 共Tret ⫺ u 䡠 ⌬E兲 ⫹ a2 䡠 Ef
(1)
to the data of each subject, with Err the reach error in
degrees, Tret the retinal location (eccentricity or disparity) of
the target, ⌬E the amount of eye displacement (version or
vergence in °), Ef the final eye position in craniotopic coordinates (version or vergence in °), and a0, a1, u, and a2 free
parameters in the fit. Parameter a0 quantifies the bias in the
99 • MAY 2008 •
www.jn.org
Downloaded from jn.physiology.org on August 11, 2008
that with those of the nonretinal model. Likewise we can ask
the question of which reach pattern is observed when targets
are presented with gaze near, but the reaches to them executed
with gaze far. Also in this case, as shown in Fig. 4E, the reach
pattern is more similar to that predicted by the retinal model
(now being the static far pattern).
To quantify these observations, we took the root of the
average of the squared difference between the error in each
dynamic trial and its predicted value by either the nonretinal or
retinal model. Consistent with the qualitative observations, this
yielded root-mean-squared error (RMSE) values lower for the
retinal model than for the nonretinal model, for both dynamic
conditions (0.8 vs. 2.2 and 0.7 vs. 2.0°, respectively). So for
these pure vergence displacements, the subject’s average reach
patterns seem to correspond best with the retinal coding
scheme.
The right panels of Fig. 4 present data of reaching movements across two directional gaze changes. Again, the two
static conditions show different patterns of reach endpoints for
different gaze directions (10° left and 10° right; Fig. 4B, left
and right, respectively). Based on these results and using the
same arguments described in the preceding text, our two
models make two different predictions about the errors in the
dynamic paradigm. Evidently, as shown by Fig. 4, D and F,
which again are based on pure version trials (see METHODS), the
observed reach patterns in the dynamic situations (left-to-right
and right-to-left gaze changes, respectively) have a much
greater similarity with the corresponding patterns predicted by
the retinal model than with the endpoint distributions predicted
by the nonretinal model. RMSE values, computed as in the
preceding text, yielded 3.1 vs. 12.7 and 4.7 vs. 14.3° (retinal vs.
nonretinal model) for the rightward and the leftward dynamic
gaze changes, respectively. This corroborates findings in previous literature that also made a clear case for the retinal
coding and updating of target direction (Henriques et al. 1998;
Medendorp and Crawford 2002; Poljac and Van den Berg
2003).
To further quantify the results of this subject (RV), we
computed RMSE values related to the retinal and nonretinal
model across all possible gaze displacement manipulations,
thus also including the trials with a combined vergence-version
change (see METHODS). In this analysis, performed in either
dimension (depth/direction), we included in one manipulation
the three reach patterns that were obtained with the eyes always
starting at the same fixation point and ending at points that
have the same vergence (or version) difference from this point,
irrespective of the version (or vergence) component. The
RMSE was then computed based on the average reach pattern
from these three refixations (always 1 pure change and 2
combined version-vergence changes). Because we used nine
initial fixation points, and two vergence (or version) differences relative to each point, this makes 18 manipulations in
total per dimension. We plotted these values versus each other
in Fig. 5, separately for the depth (A) and direction (B)
dimension. Data points above the diagonal would indicate a
preference for the nonretinal model; data below the diagonal
would be more consistent with the retinal model. For this
subject, this quantitative comparison clearly supported the
retinal model, showing a better provision to account for the
systematic depth and direction errors observed in the data.
RMSE Retinal (°)
2286
DEPTH CODING FOR REACHING MOVEMENTS
DEPTH
Nonretinal
2287
DIRECTION
Nonretinal
Retinal
B
FP Far►Near
FP Left►Right
A
Retinal
RMSE=0.6°
RMSE=1.2°
RMSE=7.0°
RMSE=1.7°
RMSE=7.3°
RMSE=2.0°
D
14
N=14 subjects
Direction
B
N=14
2.2
8
Final Eye Position
4
2
D
6
4
2
0
1.8
1.4
C
6
# Subjects
RMSE Retinal (°)
2.6
Updating Gain
0
B
Depth
A
DIRECTION
A
directional component, (P ⫽ 0.18; udirection ⫽ 1.13 ⫾ 0.36).
The fact that the updating gain u is close to 1 indicates that the
reach errors arise in relation to an updated retinal representation derived by correctly compensating for intervening eye
displacements.
To characterize the relative contribution of a retinally shifted
eye-centered target representation and final eye position to the
reach error, we computed the ratio (a1 ⫺ a2)/(a1 ⫹ a2). This
ratio would be one when the reach error depends only on the
updated target representation (a2 ⫽ 0) and would be minus one
when the error depends solely on final eye position (a1 ⫽ 0).
Figure 8, C and D, depicts this ratio separately for the depth
and direction dimensions. As shown, for target depth, the ratio
settles between the two extremes, showing a mean value
of ⫺0.10 ⫾ 0.27, indicating that both final eye position and an
updated disparity representation contributed about equally to
1.8
2.2
RMSE Nonretinal (°)
2.6
2
8
14
RMSE Nonretinal (°)
FIG. 7. Comparison of the mean RMS error value (across all trials) for the
retinal and nonretinal model for all subjects separately. In each subject, the
retinal model performed best for both depth (A) and direction (B) updating as
visualized by all respective data points lying below the diagonal.
J Neurophysiol • VOL
0
1
-1
No
Updating
Perfect
Updating
Final Eye
Position
0
1
Updated Target
Position
FIG. 8. Updating gains and final eye position effects in depth and directional reaching. A and B: updating gains (u) are distributed around a value of
1 for both depth and direction [u ⫽ 0.91 ⫾ 0.32 (SD) and u ⫽ 1.13 ⫾ 0.36,
respectively]. C and D: contributions of the updated target position and final
eye position to the reach error, expressed by the ratio (a1 ⫺ a2)/(a1 ⫹ a2). For
depth, both contributed about equally, mean ratio: ⫺0.10 ⫾ 0.27. For direction, mean ratio: 0.65 ⫾ 0.39, meaning the error arises for the most part in
relation to an updated direction representation.
99 • MAY 2008 •
www.jn.org
Downloaded from jn.physiology.org on August 11, 2008
reach error irrespective of target location or eye position.
Parameters a1 and u characterize the error term related to the
processing of an (updated) target representation relative to the
eyes with a1 a scaling term and u the updating gain. If errors
were to arise solely at the level of target presentation, the effect
of eye displacement would be zero, thus the updating gain
would be zero; u ⫽ 0. If the errors depend on the location of
the target relative to the new eye position, this means that the
system has taken possible eye displacements into account,
which ideally requires the updating gain to be 1, thus u ⫽ 1.
Finally, fit parameter a2 in Eq. 1 quantifies the dependence of
the errors on final eye position per se.
For all subjects, we found significant correlations: 0.2 ⬍ r ⬍
0.9 (P ⬍ 0.05 for all subjects) for the depth dimension and
0.3 ⬍ r ⬍ 0.7 (P ⬍ 0.01 for all subjects) for the direction
dimension. Parameter a0 had a mean value (⫾SD) that was
significantly different from zero for depth (2.69 ⫾ 1.68, P ⬍
0.001) but not for direction (⫺1.33 ⫾ 5.11, P ⫽ 0.35). The
histograms in Fig. 8, left, show the distribution of updating
gains u across subjects for both depth and directional updating.
The data give no clear sign of differences between depth and
directional updating. Across subjects the mean updating gain
(mean ⫾ SD) is not significantly different from 1 either for the
depth component (P ⫽ 0.33; udepth ⫽ 0.91 ⫾ 0.32) or for the
# Subjects
RMSE=0.3°
RMSE=1.0°
DEPTH
FP Near►Far
FP Right►left
C
FIG. 6. Population data. A and C: depth updating:
reach patterns for 2 typical dynamic conditions (same as
Fig. 4, in black), averaged across subjects, each superimposed on the predictions of the 2 models (in gray).
Circle size, SE of each data point. The retinal model
makes the best match to the data, for both conditions. B
and D: direction updating: data in same format. Data are
most consistent with predictions of the retinal model.
2288
S. VAN PELT AND W. P. MEDENDORP
the reach errors. For the direction dimension, in contrast, the
ratio had a value of 0.65 ⫾ 0.39, indicating that the reach errors
seem to arise primarily in relation to an updated retinal representation.
Binocular versus monocular updating
DISCUSSION
Over the last few decades, many studies have investigated
how the constancy of spatial direction for motor actions is
achieved across conjugate eye-movements. In contrast, the
mechanisms involved in the maintenance of spatial depth
across disjunctive eye movements have remained largely unexplored. Here we have addressed this issue using the accuracy
of memory-guided reaching movements to visual targets
briefly presented at different depths prior to a shift of gaze.
We tested between two models of the implementation of
depth constancy: a retinal versus a nonretinal model. To make
this distinction, we exploited the fact that the accuracy of
human reaching movements toward remembered space-fixed,
but nonfoveally viewed, targets depends on the eyes’ fixation
distance, analogous to the utilization of systematic reach errors
for testing directional coding (Henriques et al. 1998). We
hypothesized that if spatial depth is stored nonretinally, the
intervening vergence shifts in the dynamic trials should
have no effect on reaching. This model predicts that reaches in
these trials should be similar to those made in static trials at the
same initial eye position but without the intervening vergence
shift. Alternatively, if depth coding is retinal, updating for the
gaze shift becomes essential and reaches should match those of
static trials performed at the final eye position under the
assumption that the sensory consequences of the gaze shift
have been perfectly taken into account (perfect updating).
With intervening gaze-shifts, the memory-guided reaches
showed an error pattern that was based on the new eye position
and on the depth of the remembered target relative to that
position (Figs. 3– 8). This suggests that target depth is recomputed after the gaze shift as would be required if the brain
encoded depth in retinal coordinates. We found the values of
J Neurophysiol • VOL
99 • MAY 2008 •
www.jn.org
Downloaded from jn.physiology.org on August 11, 2008
As a final note, throughout our analyses, we have assumed
that target direction and depth are processed as separate signals. Theoretically it may be possible that depth information is
not processed in the form of an explicit disparity signal but
rather is computed on demand on the basis of two monocular
direction representations that are stored and updated in separate
maps. To investigate this possibility, we fitted Eq. 1 in terms of
directional components only for each eye separately. On basis
of the fitted parameters, we inferred reach depth by computing
the point of intersection of the reach directions predicted based
on monocular processing. We then quantified how well the
resulting depth errors correlated with the actual, observed
errors. Performance of this description in monocular coordinates was very poor, with 0.01 ⬍ r ⬍ 0.37 and significantly
lower (paired t-test, P ⬍ 0.001 using Fisher z-transformation
for comparing correlation coefficients) than predicted by the
binocular coding scheme as shown in the preceding text. This
warrants our assumption of binocular processing in terms of
depth and direction components in the exploitation of the test
described in the preceding text.
the updating gain near unity (Fig. 8), demonstrating the persistence of a correct representation of target depth relative to
fixation across vergence eye movements. This is in line with
perceptual observations by Gonzalez at al. (1998), who reported that perceived depth of random-dot stereograms is not
affected by changes in vergence. We deduce that the systematic reach errors in the present study must therefore arise after
the updating stage in the subsequent visuomotor transformation
from this updated retinal representation to the arm-centered
representation for reaching (Khan et al. 2005). The latter is
further emphasized by the influence of final eye position on the
reach error (Fig. 8).
As far as we are aware, no other studies have investigated
the reference frame in depth constancy of across vergence eye
movements. Krommenhoek and Van Gisbergen (1994), who
tested human subjects in double-step eye movement experiments with combined version-vergence movements, reported
that the saccadic and vergence system can use nonretinal
feedback about a prior eye movement in direction and depth.
They did not, however, address the spatial representation that
underlies spatial constancy in this behavior.
All subjects tested (n ⫽ 14) favored the retinal model of
depth coding. This result is warranted by the fact that our
control results on spatial direction, obtained in the same experiments, provide a strong confirmation of the earlier literature. Several behavioral studies on directional constancy have
reported evidence for retinal updating of target direction (Henriques et al. 1998; Medendorp and Crawford 2002; Van Pelt
and Medendorp 2007). Corroborating these findings, we found
the updating gain for directional updating to be close to one
(see Fig. 8). Moreover, several monkey and human brain areas
show activity related to the retinal coding and updating of the
direction of remembered targets, including frontal (Goldberg
and Bruce 1990) and parietal areas (Batista et al. 1999; Duhamel et al. 1992; Medendorp et al. 2003a). The present study
suggests that target depth is also coded and updated in retinal
maps, presumably in the form of absolute disparity coordinates
(Cumming and DeAngelis 2001). Changes of vergence alter
the values of absolute disparities, so they must be updated to
maintain spatial constancy.
Previous neurophysiological work suggests that depth representations may be constructed in areas within occipital,
frontal, and parietal cortex (Dobbins et al. 1998; Ferraina et al.
2000, 2002; Fukushima et al. 2002; Genovesio and Ferraina
2004; Genovesio et al. 2007; Gnadt and Beyer 1998; Gnadt and
Mays 1995; Rosenbluth and Allman 2002; Sakata et al. 1997;).
For example, Gnadt and Mays (1995) described neurons in the
lateral intraparietal area (LIP) of the macaque that have threedimensional receptive fields. Activity of these neurons is expressed as a function of spatial parameters in the frontoparallel
plane (horizontal and vertical eccentricity) and the relative
depth from the plane of fixation (retinal disparity). Also
Genovesio and Ferraina (2004) found LIP neurons that were
sensitive to the retinal disparity of a target but further showed
that this disparity tuning is modulated by fixation distance. A
brief report from Bhattacharyya et al. (2005) on reaching in
depth also suggest that neural activity in the parietal cortex
reflects distance to target and vergence angle. Given these
signals, it has been argued that the parietal cortex plays a role
in the integration of retinal and extraretinal information to
determine the egocentric distance of a target located in three-
DEPTH CODING FOR REACHING MOVEMENTS
J Neurophysiol • VOL
two monocular signals is less consistent with our data, demonstrated by low correlations.
That being said, in more complex updating conditions, depth
and directional signals must interact to preserve spatial constancy in retinal coordinates (Ferraina et al. 2000; Li and
Angelaki 2005; Medendorp et al. 2002, 2003b; Van Pelt and
Medendorp 2007). For example, when the body translates,
correct updating in a retinal frame requires updating to vary
from object to object, depending nonlinearly on their depth and
direction (Li and Angelaki 2005; Medendorp et al. 2003b).
Recently we showed, using memory-guided reaching movements, that the updating of target direction for translational
motion is compromised by small errors, which increase with
depth from fixation and reverse in direction for opposite depths
from fixation (Van Pelt and Medendorp 2007), consistent with
translational updating in retinal coordinates. Li and Angelaki
(2005) reported that monkeys can update target distance during
body motion in depth using extraretinal vestibular information.
Borne out by the present results, we propose that these vestibular signals interact with retinal disparity and eccentricity
information to retain 3-D stability during body motion in
space.
ACKNOWLEDGMENTS
The authors thank Dr. R. H. Cuijpers for valuable comments on an earlier
version of the manuscript.
GRANTS
This work was supported by grants from the Netherlands Organization for
Scientific Research (VIDI: 452-03-307) and the Human Frontier Science
Program (CDA) to W. P. Medendorp.
REFERENCES
Andersen RA, Essick GK, Siegel RM. Encoding of spatial location by
posterior parietal neurons. Science 230: 456 – 458, 1985.
Baker JT, Harper TM, Snyder LH. Spatial memory following shifts of gaze.
I. Saccades to memorized world-fixed and gaze-fixed targets. J Neurophysiol 89: 2564 –2576, 2003.
Batista AP, Buneo CA, Snyder LH, Andersen RA. Reach plans in eyecentered coordinates. Science 285: 257–260, 1999.
Baud-Bovy G, Viviani P. Pointing to kinesthetic targets in space. J Neurosci
18: 1528 –1545, 1998.
Beurze SM, Van Pelt S, Medendorp WP. Behavioral reference frames for
planning human reaching movements. J Neurophysiol 96: 352–362, 2006.
Bhattacharyya R, Musallam S, Andersen RA. Reaching in depth: neural
activity in posterior parietal cortex reflects distance to target and vergence
angle. Soc Neurosci Abstr 289: 18, 2005.
Bock O. Contribution of retinal versus extraretinal signals towards visual
localization in goal-directed movements. Exp Brain Res 64: 476 – 482, 1986.
Colby CL, Goldberg ME. 1999. Space and attention in the parietal cortex.
Annu Rev Neurosci 22: 319 –349, 1999.
Cumming BG, DeAngelis GC. The physiology of stereopsis. Annu Rev
Neurosci 24: 203–238, 2001.
DeAngelis GC. Seeing in three dimensions: the neurophysiology of stereopsis.
TICS 4: 80 –90, 2000.
Dobbins AC, Fiser J, Allman JM. Distance modulation of neural activity in
the visual cortex. Science 281: 552–555, 1998.
Duhamel JR, Colby CL, Goldberg ME. The updating of the representation
of visual space in parietal cortex by intended eye movements. Science 255:
90 –92, 1992.
Ferraina S, Paré M, Wurtz RH. Disparity sensitivity of frontal eye field
neurons. J Neurophysiol 83: 625– 629, 2000.
Ferraina S, Paré M, Wurtz RH. Comparison of cortico-cortical and corticocollicular signals for the generation of saccadic eye movements. J Neurophysiol 87: 845– 858, 2002.
Flanders M, Tillery SI, Soechting JF. Early stages in a sensorimotor
transformation. BBS 15: 309 –362, 1992.
99 • MAY 2008 •
www.jn.org
Downloaded from jn.physiology.org on August 11, 2008
dimensional (3-D) space (Genovesio and Ferraina 2004). It
remains to be investigated whether the computation of egocentric depth is an automated process or is enforced on demand
only when a (reach) action is prepared. Cumming and DeAngelis (2001) indicated that the updating of target distance may
be expressed by changes in retinal disparity representations.
Recently Genovesio et al. (2007) recorded neural activity in
LIP while monkeys performed saccades between targets in
different depths. They showed that in the postsaccadic period,
neural activity is influenced conjunctively by both the eye
displacement and the new eye position. In this respect, the
behavioral observations made in the present study indicate a
striking correspondence by showing effects of the same types
of signals on the depth component of the reaching errors. It can
be argued that these signals play a role in the dynamic retinal
representation of visual space and in the further transformation
of spatial information in other coordinates systems (Andersen
et al. 1985; Genovesio et al. 2007).
In support of this dynamic spatial representation, it has also
been shown that neurons in LIP actually begin to respond
before the eye movement to stimuli that will enter the receptive
field after the eye movement (Duhamel et al. 1992; Nakamura
and Colby 2002). In other words, LIP neurons anticipate the
sensory consequences of the future eye position before the
saccade is executed, which suggest that the updating mechanisms relies on a copy of the eye motor command (Sommer
and Wurtz 2002) rather than on sensory feedback that arrives
much later. A useful experiment to be performed in this context
would be to investigate if predictive updating also occurs in
relation to vergence eye movements. Along these lines, Kaiser
and Lappe (2004) reported recently that visual objects flashed
shortly before or during a saccade are mislocalized, resembling
a compression of space around the saccade target. They attributed this distortion to the remapping process in parietal cortex,
and it would be interesting to see whether similar spatial
distortions occur across vergence eye movements, and if so,
whether they have a similar time course. Viewed from a
different perspective, a recent hypothesis here, put forward by
Vaziri et al. (2006), is that the brain integrates the predicted
sensory consequences of motor commands with the actual sensory (feedback) information to produce an estimate of sensory
space that is better than possible from either source alone. More
experiments are required to see if this hypothesis is upheld
across combined saccade-vergence movements or in further
extended conditions involving movements of the head and
body in space.
Finally, it is important to emphasize that the present study
considered direction and depth as independent spatial variables, processed and updated separately during updating across
eye movements. We made this assumption based on results of
various reaching studies, performed in static conditions, showing that the spatial distributions of movement endpoints of
reaches toward remembered targets were elliptical in shape with a
tendency of the major axis to be directed to the subject’s eyes
(Baud-Bovy and Viviani 1998; Henriques et al. 2003; McIntyre
et al. 1997). This implies that noise in the reach is larger for the
depth than for the directional component in these cases, suggesting that both dimensions are controlled separately. The
present analyses supported this assumption by demonstrating
that an alternative, implicit depth representation emerging from
2289
2290
S. VAN PELT AND W. P. MEDENDORP
J Neurophysiol • VOL
Medendorp WP, Smith MA, Tweed DB, Crawford JD. Rotational remapping in human spatial memory during eye and head motion. J Neurosci 22:
196RC, 2002.
Medendorp WP, Tweed DB, Crawford JD. Motion parallax is computed in
the updating of human spatial memory. J Neurosci 23: 8135– 8142, 2003b.
Medendorp WP, Van Asselt S, Gielen CC. Pointing to remembered visual
targets after active one-step self-displacements within reaching space. Exp
Brain Res 125: 50 – 61, 1999.
Mergner T, Nasios G, Maurer C, Becker W. Visual object localisation in
space. Exp Brain Res 141: 33–52, 2001.
Merriam EP, Genovese CR, Colby CL. Spatial updating in human parietal
cortex. Neuron 39: 361–373, 2003.
Merriam EP, Genovese CR, Colby CL. Remapping in human visual cortex.
J Neurophysiol 97: 1738 –1755, 2007.
Nakamura K, Colby CL. Updating of the visual representation in monkey
striate and extrastriate cortex during saccades. Proc Natl Acad Sci USA 99:
4026 – 4031, 2002.
Philbeck JW, Loomis JM. Comparison of two indicators of perceived
egocentric distance under full-cue and reduced-cue conditions. JEP: Hum
Perception Performance 23: 72– 85, 1997.
Poljac E, Van den Berg AV. Localization of the plane of regard in space. Exp
Brain Res 163: 457– 467, 2003.
Rosenbluth D, Allman JM. The effect of gaze angle and fixation distance on
the responses of neurons in V1, V2, and V4. Neuron 33: 143–149, 2002.
Sakata H, Taira M, Kusunoki M, Murata A, Tanaka Y. The Trends
Neurosci Lecture. The parietal association cortex in depth perception and
visual control of hand action. Trends Neurosci 20: 350 –357, 1997.
Schlag J, Schlag-Rey M, Dassonville P. Saccades can be aimed at the spatial
location of targets flashed during pursuit. J Neurophysiol 64: 575–581, 1990.
Shadmehr R, Wise SP. The Computational Neurobiology of Reaching and
Pointing: A Foundation for Motor Learning. Cambridge, MA: MIT Press,
2005.
Sommer MA, Wurtz RH. A pathway in primate brain for internal monitoring
of movements. Science 296: 1480 –1482, 2002.
Sparks DL, Mays LE. Spatial localization of saccade targets. I. Compensation
for stimulation-induced perturbations in eye position. J Neurophysiol 49:
45– 63, 1983.
Van Pelt S, Medendorp WP. Gaze-centered updating of remembered visual
space during active whole-body translations. J Neurophysiol 97: 1209 –
1220, 2007.
Van Pelt S, Van Gisbergen JA, Medendorp WP. Visuospatial memory
computations during whole-body rotations in roll. J Neurophysiol 94:
1432–1442, 2005.
Vaziri S, Diedrichsen J, Shadmehr R. Why does the brain predict sensory
consequences of oculomotor commands? Optimal integration of the predicted and the actual sensory feedback. J Neurosci 26: 4188 – 4197, 2006.
Vindras P, Desmurget M, Viviani P. Error parsing in visuomotor pointing
reveals independent processing of amplitude and direction. J Neurophysiol
94: 1212–1224, 2005.
Walker MF, Fitzgibbon J, Goldberg ME. Neurons of the monkey superior
colliculus predict the visual result of impeding saccadic eye movements.
J Neurophysiol 73: 1988 –2003, 1995.
Wei M, DeAngelis GC, Angelaki DE. Do visual cues contribute to the neural
estimate of viewing distance used by the oculomotor system? J Neurosci 23:
8340 – 8350, 2003.
99 • MAY 2008 •
www.jn.org
Downloaded from jn.physiology.org on August 11, 2008
Fukushima K, Yamanobe T, Shinmei Y, Fukushima J, Kurkin S, Peterson
BW. Coding of smooth eye movements in three-dimensional space by
frontal cortex. Nature 419: 157–162, 2002.
Genovesio A, Brunamonti E, Giusti MA, Ferraina S. Postsaccadic activities
in the posterior parietal cortex of primates are influenced by both eye
movement vectors and eye position. J Neurosci 27: 3268 –3273, 2007.
Genovesio A, Ferraina S. Integration of retinal disparity and fixation-distance
related signals toward an egocentric coding of distance in the posterior
parietal cortex of primates. J Neurophysiol 91: 2670 –2684, 2004.
Gnadt JW, Beyer J. Eye movements in depth: what does the monkey’s
parietal cortex tell the superior colliculus? Neuroreport 9: 233–238, 1998.
Gnadt JW, Mays LE. Neurons in monkey parietal area LIP are tuned for
eye-movement parameters in three-dimensional space. J Neurophysiol 73:
280 –297, 1995.
Goldberg ME, Bruce CJ. Primate frontal eye fields. III. Maintenance of a
spatially accurate saccade signal. J Neurophysiol 64: 489 –508, 1990.
Gonzalez F, Rivadulla C, Perez R, Cadarso C. Depth perception in random
dot stereograms is not affected by changes in either vergence or accommodation. Optom Vis Sci 75: 743–747, 1998.
Hallett PE, Lightstone AD. Saccadic eye movements towards stimuli triggered by prior saccades. Vision Res 16: 99 –106, 1976.
Henriques DY, Crawford JD. Direction-dependent distortions of retinocentric space in the visuomotor transformation for pointing. Exp Brain Res 132:
179 –194, 2000.
Henriques DY, Klier EM, Smith MA, Lowy D, Crawford JD. Gazecentered remapping of remembered visual space in an open-loop pointing
task. J Neurosci 18: 1583–1594, 1998.
Henriques DY, Medendorp WP, Gielen CC, Crawford JD. Geometric
computations underlying eye-hand coordination: orientations of the two
eyes and the head. Exp Brain Res 152: 70 –78, 2003.
Howard IP, Rogers BJ. Binocular Vision and Stereopsis. New York: Oxford
Claridon, 1995.
Israel I, Chapuis N, Glasauer S, Charade O, Berthoz A. Estimation of
passive horizontal linear whole-body displacement in humans. J Neurophysiol 70: 1270 –1273, 1993.
Julesz B. Foundations of Cyclopean Perception. Chicago: Chicago University
Press, 1971.
Khan AZ, Pisella L, Rossetti Y, Vighetto A, Crawford JD. Impairment of
gaze-centered updating of reach targets in bilateral parietal-occipital damaged patients. Cereb Cortex 15: 1547–1560, 2005.
Kaiser M, Lappe M. Perisaccadic mislocalization orthogonal to saccade
direction. Neuron 41: 293–300, 2004.
Klier EM, Angelaki DE, Hess BJM. Roles of gravitational cues and efference
copy signals in the rotational updating of memory saccades. J Neurophysiol
94: 468 – 478, 2005.
Krommenhoek KP, Van Gisbergen JA. Evidence for nonretinal feedback in
combined version-vergence eye movements. Exp Brain Res 102: 95–109,
1994.
Li N, Angelaki DE. Updating visual space during motion in depth. Neuron 48:
149 –158, 2005.
McIntyre J, Stratta F, Lacquaniti F. A viewer-centered reference frame for
pointing to memorized targets in three-dimensional space. J Neurophysiol
78: 1601–1618, 1997.
Medendorp WP, Crawford JD. Visuospatial updating of reaching targets in
near and far space. Neuroreport 13: 633– 636, 2002.
Medendorp WP, Goltz HC, Vilis T, Crawford JD. Gaze-centered updating
of visual space in human parietal cortex. J Neurosci 23: 6209 – 6214, 2003a.

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project