Stan Van Pelt and W. Pieter Medendorp J Neurophysiol 99:2281-2290, 2008. First published Mar 19, 2008; doi:10.1152/jn.01281.2007 You might find this additional information useful... This article cites 55 articles, 32 of which you can access free at: http://jn.physiology.org/cgi/content/full/99/5/2281#BIBL Updated information and services including high-resolution figures, can be found at: http://jn.physiology.org/cgi/content/full/99/5/2281 Additional material and information about Journal of Neurophysiology can be found at: http://www.the-aps.org/publications/jn This information is current as of August 11, 2008 . Downloaded from jn.physiology.org on August 11, 2008 Journal of Neurophysiology publishes original articles on the function of the nervous system. It is published 12 times a year (monthly) by the American Physiological Society, 9650 Rockville Pike, Bethesda MD 20814-3991. Copyright © 2005 by the American Physiological Society. ISSN: 0022-3077, ESSN: 1522-1598. Visit our website at http://www.the-aps.org/. J Neurophysiol 99: 2281–2290, 2008. First published March 19, 2008; doi:10.1152/jn.01281.2007. Updating Target Distance Across Eye Movements in Depth Stan Van Pelt1 and W. Pieter Medendorp1,2 1 Nijmegen Institute for Cognition and Information, 2FC Donders Centre for Cognitive Neuroimaging, Radboud University Nijmegen, Nijmegen, The Netherlands Submitted 23 November 2007; accepted in final form 16 March 2008 INTRODUCTION Maintaining spatial constancy across self-generated movements is crucial for veridical perception of the world and for accurate control of goal-directed actions. Over the past few decades, the quality of spatial constancy has been investigated systematically across various types of self-motion, including eye, head, and body movements. As a result, it is now well established that spatial constancy is preserved across intervening saccadic (Hallet and Lightstone 1976; Sparks and Mays 1983) and smooth pursuit eye movements (Baker et al. 2003; Schlag et al. 1990). Also a reorientation of the head or displacement of the body in space does not compromise spatial stability to a great extent (Israel et al. 1993; Li and Angelaki 2005; Medendorp et al. 1999, 2002, 2003b; Mergner et al. 2001; Van Pelt and Medendorp 2007; Van Pelt et al. 2005). From a mechanistic perspective, there has been considerable debate over how the brain solves the spatial constancy problem. In the absence of allocentric cues, it seems that an egocentric, gaze-centered reference frame dominates in the mechanisms of spatial stability for simple saccade or reaching Address for reprint requests and other correspondence: W. P. Medendorp, Nijmegen Institute for Cognition and Information, Radboud University Nijmegen, P.O. Box 9104, NL-6500 HE, Nijmegen, The Netherlands (E-mail: [email protected]). www.jn.org tasks (Henriques et al. 1998; Klier et al. 2005; Medendorp and Crawford 2002; Van Pelt and Medendorp 2007). In support, cells in monkey extrastriate visual areas (Nakamura and Colby 2002), posterior parietal cortex (Batista et al. 1999; Colby and Goldberg 1999; Duhamel et al. 1992), frontal cortex (Goldberg and Bruce 1990), and superior colliculus (Walker et al. 1995), as well as in the human posterior parietal cortex (Medendorp et al. 2003a; Merriam et al. 2003, 2007) have been shown to update the gaze-centered coordinates of remembered stimuli to maintain an accurate representation of visual space across saccades. Notwithstanding these convincing observations and clear insights, it should be emphasized that nearly all these studies were limited by only examining the directional aspect of spatial constancy. For many spatially guided actions, however, directional constancy is not the only spatial requirement; the constancy of target depth (or distance) is another essential component that should be mediated by the signals and mechanisms for spatial stability. Because it is generally assumed that target depth and direction are processed in functionally distinct visuomotor channels (Cumming and DeAngelis 2001; DeAngelis 2000; Flanders et al. 1992; Vindras et al. 2005), the mechanisms to preserve their constancy may also operate independently, at least to some extent. To date, only few studies have explicitly assessed the constancy of target depth during self-motion (Krommenhoek and Van Gisbergen 1994; Li and Angelaki 2005; Medendorp et al. 1999, 2003b; Philbeck and Loomis 1997). Krommenhoek and Van Gisbergen (1994) showed that subjects can look at a remembered position of a target in depth after a vergence eye movement. Li and Angelaki (2005) reported that non-human primates can keep track of changes in the distance of nearby objects when their body moved toward or away from them. Despite these quantitative observations, the computational mechanisms underlying depth constancy have not been addressed. The objective of the present study is to fill this lacuna by testing between two models for depth coding in the visuomotor system. While a variety of cues to depth can be used by the visual system, binocular disparity dominates in the creation of a cohesive, three-dimensional depth perception (Howard and Rogers 1995; Julesz 1971; Wei et al. 2003). Binocular disparity is caused by the slight difference in viewpoint of the two eyes due to their differential location in the head. Objects at different distances from the eyes’ fixation distance project onto different positions on each retina and thus cause different horizontal binocular disparities. Likewise, a single object at a The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact. 0022-3077/08 $8.00 Copyright © 2008 The American Physiological Society 2281 Downloaded from jn.physiology.org on August 11, 2008 Van Pelt S, Medendorp WP. Updating target distance across eye movements in depth. J Neurophysiol 99: 2281–2290, 2008. First published March 19, 2008; doi:10.1152/jn.01281.2007. We tested between two coding mechanisms that the brain may use to retain distance information about a target for a reaching movement across vergence eye movements. If the brain was to encode a retinal disparity representation (retinal model), i.e., target depth relative to the plane of fixation, each vergence eye movement would require an active update of this representation to preserve depth constancy. Alternatively, if the brain was to store an egocentric distance representation of the target by integrating retinal disparity and vergence signals at the moment of target presentation, this representation should remain stable across subsequent vergence shifts (nonretinal model). We tested between these schemes by measuring errors of human reaching movements (n ⫽ 14 subjects) to remembered targets, briefly presented before a vergence eye movement. For comparison, we also tested their directional accuracy across version eye movements. With intervening vergence shifts, the memory-guided reaches showed an error pattern that was based on the new eye position and on the depth of the remembered target relative to that position. This suggests that target depth is recomputed after the gaze shift, supporting the retinal model. Our results also confirm earlier literature showing retinal updating of target direction. Furthermore, regression analyses revealed updating gains close to one for both target depth and direction, suggesting that the errors arise after the updating stage during the subsequent reference frame transformations that are involved in reaching. 2282 S. VAN PELT AND W. P. MEDENDORP A B DEPTH STATIC FP Far FP Left FP Near FP Right FIG. 1. Testing between nonretinal and retinal models for memory encoding of spatial depth and direction. A and B: static condition: it is supposed that the reach toward a space-fixed target will be erred depending on gaze fixation position in depth (A) and direction (B). The exact relationship is not of importance for the test. C and D: dynamic condition: a gaze shift intervenes between target presentation and reaching. The nonretinal model predicts no effect of the gaze shift on reaching. The retinal scheme requires target updating relative to the new gaze position, predicting reach errors as in the static situation with the eyes at the same final position. T C FP Far►Near Nonretinal Retinal D FP Left►Right Nonretinal Retinal ΔEye ΔEye DYNAMIC DIRECTION J Neurophysiol • VOL depicts the situation for the depth dimension when subjects changed gaze from far to near fixation after initial target perception. Reaching in depth, as in the static case without an intervening vergence eye movement (Fig. 1A, static FP far condition), would argue in favor of the use of a nonretinal depth representation. However, if the intervening eye movement leads to a depth error like that observed when the same target was viewed from the final eye position (Fig. 1A, static FP near condition), this would provide evidence for the use of an updated eye-centered binocular disparity representation. Following its original design, the test can likewise discriminate between a retinal and a nonretinal representation of target direction across saccadic eye movements (as shown by the panels in Fig. 1D). Our results suggest that the brain codes dynamic disparity and direction representations to store target locations for reaching across eye movements in depth and direction. Regression analyses revealed that these representations are modulated by using both eye position and eye displacement signals consistent with recent observations in monkey neurophysiology (Genovesio et al. 2007). METHODS Subjects Fourteen human subjects (4 female, 10 male; mean age: 26 ⫾ 4 yr) signed informed consent to participate in this study. All were free of any known sensory, perceptual, or motor disorders. Twelve participants were right-handed; two were left-handed; reaching movements were made using the preferred arm. Two subjects (the authors) were aware of the purpose of the experiments, whereas the others were naı̈ve. Experimental setup Subjects were seated in a completely darkened room with their torso securely strapped into a custom-made chair by means of two safety belts across both the torso and pelvis to minimize body movement. Their head was mechanically stabilized using a chin rest and a helmet, which was fixed to the chair by means of a frame that was adjustable in height. This ensured that only the preferred arm and the eyes could move while the rest of the body remained stationary. 99 • MAY 2008 • www.jn.org Downloaded from jn.physiology.org on August 11, 2008 fixed position from the eyes will have different horizontal disparities for different viewing distances. In this study, we investigated how the brain codes the distance of a remembered space-fixed target during intervening changes of the binocular fixation point (i.e., vergence eye movements). We reasoned that if the brain was to encode a binocular disparity representation, i.e., target depth relative to the eyes’ fixation point (Shadmehr and Wise 2005), each disjunctive change of gaze will require an active update of this representation to maintain spatial constancy. Alternatively, if the brain was to store a nonretinal depth representation of the target by integrating binocular disparity and vergence signals at the moment of target presentation (Genovesio and Ferraina 2004; Genovesio et al. 2007), this representation should remain stable for subsequent vergence eye movements. To test between these hypotheses, we employed a memoryguided reach paradigm adopted from Henriques et al. (1998), who originally developed it to examine the computations for directional spatial constancy. We expanded this test by examining actual versus predicted localization errors in depth when vergence eye movements intervene between viewing a target and reaching toward its remembered location as will be further outlined in Fig. 1. The assumption behind our test was that subjects make systematic distance errors in their reach toward memorized targets, depending on their fixation depth (static reaching, Fig. 1A)—as they have shown to make directional errors depending on their gaze direction (Fig. 1B) (Henriques et al. 1998). In the latter case, the phenomenon has been termed the “retinal exaggeration effect” because subjects tend to overshoot the target relative to current gaze direction although individual subjects show considerable variations in this pattern (Bock 1986; Henriques and Crawford 2000). It is not known if a similar overshoot effect occurs for depth; however, as long as distance errors depend on fixation depth, even if only in a complex and idiosyncratic manner, this relationship can be exploited to distinguish between retinal and nonretinal target representations. The critical part of the test is based on the errors that occur when subjects reach after an intervening eye movement toward the location of a target that was viewed only before this eye movement (dynamic situation, Fig. 1, C and D). Figure 1C DEPTH CODING FOR REACHING MOVEMENTS 4.1 – 6.5 A 2.5 – 4 2 – 2.5 1.5 – 2 FP2 0 - 1.5 T FP1 (s) Time B Stimulus Configuration Version Change Vergence Change Combined Change 8° 10° 0° -10° ° 13 8° 1 10cm FIG. 2. Experimental paradigm. A: sequence of stimuli and the subject’s instructions. A trial started with the illumination of a red fixation light (FP1). Then, after a delay of 1.5 s, a green target (T) was cued for 0.5 s. After a further 0.5 s, the subject had to change fixation to fixation light 2 (FP2) in dynamic trials. In static trials, fixation was to be kept at FP1, thus FP2 ⫽ FP1. Next, 1.5 s later, FP2 was extinguished, and an auditory cue instructed the subject to reach toward the remembered location of the target, while keeping fixation at the remembered location of FP2. Open circles, possible stimulus locations; filled circles, exemplar stimuli presented. B: potential locations of the stimuli, which served either as (initial or final) fixation point or memory target, or both, were on the intersections of 3 isoversion (⫺10, 0, and 10°) and 3 isovergence (8, 13, and 18°) lines. In dynamic trials, gaze displacements could consist of a pure version movement (2nd panel), a pure vergence change (3rd panel), or a combination of both (rightmost panel). J Neurophysiol • VOL Optotrak data were sampled at 125 Hz with an accuracy of ⬍0.2 mm and saved on a PC for off-line analysis. We recorded the subjects’ binocular eye movements using an Eyelink II eyetracker (SR Research) mounted to the chair-fixed helmet. This system tracks the pupils’ positions using infrared light reflection at a sampling rate of 250 Hz. Before the experiment began, eye movements were calibrated by fixating the stimulus LEDs three times each, in complete darkness. This resulted in a calibration accuracy ⬍0.5°. Calibration was checked off-line, to allow for drift correction due to headband slippage or other factors. Because the head and body stayed fixed during the experiment, the orientation of the eyes within the head, as measured by the tracker, was equivalent to the orientation of the eyes in space (gaze). Rightward rotations were taken as positive. Two PCs in a master-slave arrangement controlled the experiment. The master PC contained hardware for data acquisition of the Optotrak measurements and visual stimulus control. The slave PC was equipped with hardware and software from the Eyelink system. Experimental paradigm The main focus of this study is to reveal the reference frame employed by the brain to maintain spatial constancy for depth. To allow for comparison with previous studies (Beurze et al. 2006; Henriques et al. 1998; Medendorp and Crawford 2002), we employed a paradigm that also tested the mechanisms for directional constancy. Figure 2A illustrates the paradigm. A trial started with the onset of a red fixation LED, which we refer to as FP1 (fixation point 1), to be fixated for its entire illumination duration of 2.5 s. FP1 could be any of the nine stimulus locations on the stimulus array (Fig. 2B, leftmost panel). At 1.5 s after the onset of FP1, a target for memory (T, a green LED) was flashed for 0.5 s while the subject kept gaze fixed at FP1. Thus T was on the fovea when presented at the same location of FP1 but on the peripheral retina for any of the eight other possible locations. Next, 0.5 s after the flash, a time interval of 1.5 s followed during which the subject either changed gaze fixation to a second illuminated fixation light (FP2— dynamic paradigm) or maintained fixation of the first fixation point when FP2 ⫽ FP1 (static paradigm). Subsequently, at FP2 offset (4.0 s after trial onset), the stimulus array was retracted, followed 100 ms later by an auditory signal that cued the subject to reach to T, while keeping gaze fixed at (the remembered location of) FP2. The subject had to hold the reaching position until the end of a 2.4-s interval, indicated by a second auditory signal. Then the next trial started, with FP1 at a different location than the location of T in the preceding trial, to avoid any visual feedback about performance in the previous trial. Between trials, subjects had their reaching arms resting unencumbered on their lap with the hand close to their knees. FP1, T, and FP2 were pseudorandomly selected from the stimulus array, such that all combinations of FP1, T, and FP2 were tested once. This yielded a total of 729 unique trials: 81 trials were pure static trials (FP1 ⫽ FP2) and the other trials were dynamic trials. Of the dynamic trials, 162 trials had a pure conjugate change in eye position (version eye movement), whereas vergence (disjunctive part) remained constant (Fig. 2B, 2nd panel); 162 trials had a vergence change but constant version (ignoring the small vertical version eye movements due to the vertical offset in the positioning of near and far LEDs; 3rd panel ); and 324 trials had a combined vergence-version change (rightmost panel ). The total experiment was divided into three sessions, each of which lasted for ⬃60 min each and were tested on different days. In each session, subjects performed blocks of 15 or 16 consecutive trials between which a brief rest was provided with the room lights on to avoid dark adaptation. During the experiments, subjects never received feedback about their performance. Before the actual experiments, subjects practiced a few blocks to become familiar with the task. 99 • MAY 2008 • www.jn.org Downloaded from jn.physiology.org on August 11, 2008 The stimulus array (see Fig. 2B, left) consisted of nine lightemitting diodes (LEDs), each 3 mm in diameter, and each could be flashed in two different colors, either as a green or a red light (luminance ⬍20 mcd/m2). The LEDs had fixed positions on a frame, which could be moved by a robotic arm. Stimuli were presented in front of the subject, in a horizontal plane, slightly below the eyes, at the intersections of three imaginary horopters (equal vergence lines: 8, 13, and 18° vergence, i.e., 46.5, 28.5, and 20.5 cm from the subjects eyes) and three equidirection lines (⫺10, 0, and ⫹10° version), based on an average interocular distance of 6.5 cm. The robotic arm was equipped with stepping motors (type Animatics SmartMotors, Servo Systems) and could rapidly move the stimulus array to various positions within the workspace, bringing it within 200 ms out of touch during the reaching task performed by the subject (see following text) (see also Van Pelt and Medendorp 2007). During the experiments, the total movement time of the robot was always 2.3 s. Also between trials, when the room lights were on, the stimulus array was close to the ceiling of the experimental room. This way only the frame’s rear side could be viewed, which this gave no information about the spatial configuration of the stimuli. Prior to the experiments, we measured the location of the eyes in space and the locations of the space-fixed stimulus LEDs using an Optotrak Certus system (Northern Digital). With this information, we were able to compute the direction and distance of the stimulus LEDs with respect to the subject’s eyes. During the experiment, the Optotrak continuously recorded the location of the tip of the index finger. We ensured that the fingertip was at least always visible during the last part of the reaching movement (see Van Pelt and Medendorp 2007). 2283 S. VAN PELT AND W. P. MEDENDORP Data analysis DEPTH RESULTS The experiments were designed to test between retinal and nonretinal models of the coding of target depth and direction. The basic premise for this test is a difference in accuracy of reaching movements toward remembered space-fixed but nonfoveally viewed targets for different eye fixation positions (Beurze et al. 2006; Henriques et al. 1998). The two models make clearly different predictions about the reach errors that would arise when a gaze displacement intervenes between seeing the target and reaching to its remembered location (dynamic condition). The nonretinal model predicts an error similar to that observed in the static condition without the intervening gaze displacement, whereas the retinal model predicts an error similar to that observed for a target viewed from the same final fixation position (see Fig. 1). Task performance Figure 3 illustrates task performance of a typical subject, demonstrating static and dynamic trials testing either for depth constancy (left) or for direction constancy (right). The left panels show binocular vergence (gray traces) and measured fingertip depth (black traces) over the time course of eight trials J Neurophysiol • VOL B Far Fixation Left Fixation 20 20 15 10 5 0 10 0 -10 T R FP1 T 20 15 10 5 0 Right Fixation T C DYNAMIC R FP1 R FP1 -20 Near Fixation Vergence (°) STATIC A 20 10 0 -10 T D Far ►Near R FP1 -20 Left ►Right 20 20 15 10 5 0 10 0 -10 T FP1 0 R FP2 2 4 T FP1 -20 0 6 R FP2 2 4 6 Time (s) Eye Finger Ideal Eye Target FIG. 3. Typical performance of 1 subject. Eye position (version, vergence, in gray), target (dotted), fingertip position (in black), expressed in binocular depth and direction coordinates (in deg), plotted as function of time. Dashed trace, geometrically ideal eye position. Fingertip traces are only shown for the last 1.5 s of the reach period. Left panels: investigation of depth coding of a target at 13° vergence angle, with 2 static conditions (far vs. near fixation, A), and the corresponding dynamic condition (far to near, C). Right panels: investigation of directional coding of a craniotopically central target with 2 static conditions (left vs. right fixation, B), and the corresponding dynamic condition (D). Thin boxes, time intervals of the different trial stages [target presentation (T), FP1 and FP2 periods, reach interval (R)]. with a target flashed at the middle depth (13° vergence when foveated, dotted line, see also Fig. 2B). Following instructions, the eyes fixated at either one of the far targets (Fig. 3A, top, requiring a smaller vergence angle) or at one of the near targets (Fig. 3A, top, requiring a larger vergence angle) or reoriented from far to near fixation after stimulus presentation (Fig. 3C). In all conditions, final eye fixation, which was to be maintained during the reach, showed a small decline in vergence after the offset of the fixation point, at the go cue for the reach, but note that reach responses were performed in complete darkness. In the static trials, reaches (black) showed small overshoots (smaller response angle than required) depending on the eyes’ fixation depth, with errors of about ⫺2.2 ⫾ 0.8° and ⫺4.5 ⫾ 0.9° for far and near fixation. In the dynamic trials, in which the target is viewed with far fixation and the reach is performed with the eyes fixating near (Fig. 3C), the errors seem qualitatively indistinguishable from those in the static near situation (Fig. 3A, bottom) with a mean error of ⫺4.7 ⫾ 0.7°. Thus a change of gaze in depth affects the reaching responses to previously seen targets, making them look like those with gaze stationary at the same final depth. Figure 3, B and D, shows eight typical time courses in each of three trial types serving to illustrate how the directional coding of a craniotopically central target depends on (changes in) 99 • MAY 2008 • www.jn.org Downloaded from jn.physiology.org on August 11, 2008 Data were analyzed off-line using Matlab (The Mathworks). Optotrak data were first transformed to a right-handed Cartesian coordinate system, referenced to the position of the cyclopean eye. In this coordinate system, the positive y axis pointed leftward along the shoulder line (from the subject’s perspective), the x axis pointed forward and the z axis upward. Horizontal gaze direction was computed for each eye separately; binocular version and vergence angles were calculated from the left (L) and right (R) gaze directions as (R ⫹ L)/2 and L ⫺ R, respectively. Rightward rotations were taken as positive. Cartesian positions of FP1, FP2, T, and the fingertip were also expressed in binocular coordinates, in terms of depth and direction (in degrees) from the cyclopean eye. This allowed for the computation of reach and target depth relative to the plane of fixation, expressed in terms of angular disparity (Howard and Rogers 1995). By convention, crossed disparities were taken as positive. We discarded trials in which subjects did not maintain fixation within a 5 ⫻ 4° (version ⫻ vergence) interval around the fixation points or made a saccade during target presentation. For the remaining trials, eye fixation accuracy was 2.60 ⫾ 0.83° (mean ⫾ SD). We also excluded trials in which subjects had not correctly followed the reaching instructions of the paradigm, i.e., when they started their reaching movement too early or did not adopt a stable reach position during the response intervals (fingertip velocity ⬎5 cm/s based on Optotrak data). Overall ⬍3% of the trials was discarded on the basis of these arm and eye movement criteria. The endpoint of each reaching movement was selected at the time at which the velocity of the fingertip first dropped below 5 cm/s within the 2.4-s reaching interval, under the requirement that the arm had correctly followed the instructions of the paradigm. An average position was computed over an eight-sample interval (64 ms) centered at this time point. Analyses were performed separately for the directional and depth dimensions. We assessed performance by quantifying reach errors in both dimensions for each trial. Using multiple linear regression analysis, we investigated the effects of eye displacement and eye position on the reach errors that were observed. Statistical tests were performed at the 0.05 level (P ⬍ 0.05). DIRECTION Version (°) 2284 DEPTH CODING FOR REACHING MOVEMENTS gaze direction. Again, in all trials, the eyes act according to instructions, showing steady fixation during target presentation and only small saccades at the moment of reach. Reaching behavior in the dynamic trials, in which gaze changes from 10° leftward to 10° rightward direction (Fig. 3D), matches more closely the observations made in static trials with gaze in the 10° rightward direction than with gaze at the same direction of stimulus presentation (Fig. 3B). For the static trials, mean horizontal reach error was 2.5 ⫾ 2.8 and ⫺12.1 ⫾ 3.5° in the static left and right conditions and ⫺11.5 ⫾ 3.4°) in the dynamic condition. In other words, the change in eye position has had a marked effect on reaching behavior. Reach patterns DEPTH FP Far DIRECTION FP Near B FP Left FP Right STATIC A portantly, reach patterns in the two static conditions seem noticeably different, depending on the eyes’ fixation depth. The question is which of these reach patterns is observed in a dynamic condition in which the subject viewed the target with far fixation, then changed gaze toward near fixation and subsequently reached toward the remembered target location. If the subject had stored absolute target depth relative to the body (nonretinal model), computed at the time of seeing the target, the intervening gaze deviation should have no systematic effect on the reach responses. Thus the nonretinal model predicts reach errors as in the static condition with gaze in far space. However, if the subject had stored target depth relative to the plane of fixation, as a retinal disparity signal (retinal model), this signal must be updated for the gaze change, predicting an error pattern similar to that observed in the static condition with near fixation. Figure 4C shows the systematic reach patterns obtained in the related dynamic trials (thick black lines), superimposed on the predictions of either of the two models. Note that we based this illustration on “pure” trials only, i.e., trials with a change in vergence (far-to-near fixation) but constant version (see METHODS), to demonstrate the effect of the vergence change in the clearest possible fashion. It is important to realize that using “combined” trials here (trials with a change in both vergence and version, see METHODS) could easily obscure the main effects in either dimension. Clearly for this subject and when gaze was displaced from far to near (Fig. 4C), the data seem more consistent with the predictions of the retinal model Nonretinal Retinal Retinal FP Far►Near FP Left►Rigth D RMSE=2.2° RMSE=0.8° RMSE=12.7° RMSE=3.1° RMSE=14.3° RMSE=4.7° F FP Right►Left E FP Near►Far DYNAMIC C Nonretinal RMSE=2.0° RMSE=0.7° J Neurophysiol • VOL 99 • MAY 2008 • www.jn.org FIG. 4. Reach patterns across all targets from 1 subject (RV). A: depth coding with reach patterns (in gray) in 2 static conditions (far vs. near), superimposed on the target grid (in thin black). C and E: reach pattern in the corresponding dynamic conditions (in black), superimposed on the patterns (in gray) from A, that serve as predictions of the nonretinal and retinal models. B: directional coding with reach patterns in 2 static conditions superimposed on target grid. D and F: reach patterns in the corresponding dynamic conditions, superimposed on the predicted patterns by the 2 models. root-mean-squared error (RMSE, in deg), a measure of the deviation of the dynamic data to the model prediction. Each data point represents the mean of 3 pure trials with version as the free parameter for the depth trials (A, C, and E) and vergence for the direction trials (B, D, and F) dimension, respectively. Circle size, SE of each data point. Downloaded from jn.physiology.org on August 11, 2008 To demonstrate performance in the static and dynamic conditions more clearly, Fig. 4 shows spatial plots of reach endpoints (filled circles) for a single subject (RV). The size of each circle represents the corresponding confidence limit (see legend for computation). In the static conditions with far or near fixation (Fig. 4A), the mean reach endpoints toward the nine targets are interconnected with gray lines, and superimposed on the spatial structure defined by the stimulus locations (thin black lines). Perfect behavior would require that reach responses align with the stimulus matrix. This is clearly not the case: the subject makes substantial errors with regard to both depth and direction of nearly all target locations. Reaching movements of this subject undershoot the distance of some targets, whereas they are more accurate in others. More im- 2285 S. VAN PELT AND W. P. MEDENDORP J Neurophysiol • VOL A B Depth 4 16 N=18 manipulations Direction N=18 3 8 2 1 2 3 RMSE Nonretinal (°) 4 0 8 16 RMSE Nonretinal (°) FIG. 5. RMS errors in relation to the 2 models plotted vs. each other. Data pooled across trials that have the same gaze displacement (same dynamic situation), in 1 subject (RV, same as in Fig. 4). For both depth (A) and direction (B), most data points fall below the diagonal, indicating that the retinal model gives a better description of the data in this subject. Next, before we proceed further, recall that the efficacy of our test is based on the premise that the reach error in the static condition depends on eye position. Figures 4 and 5 confirm this for a single subject. To test this assumption in an analysis across subjects, we performed a 3 ⫻ 3 repeated-measures ANOVA on the static reach errors with eye position and craniotopic target location as within-subject factors. This analysis revealed a significant effect of either factor, in both the depth and direction dimension [in all tests F(2,12)⬎6.3, P ⬍ 0.05], which validates the basic premise of our test. Subsequently, under this confirmed assumption, Fig. 6 quantifies the results of all 14 subjects in two typical dynamic conditions (far-to-near, near-to-far and left-to-right, right-toleft). Analogous to the observations made in Fig. 4, the figure demonstrates that the retinal model fits the mean pattern of reach errors across subjects better than the nonretinal model (compare the corresponding RMSE values). The mean RMSE values across all testing conditions are given in Fig. 7, A and B, for each subject separately. Across our population of subjects, the retinal model produced the best description for the coding of both target distance and direction (paired t-test, P ⬍ 0.001). Model analysis Although the data of our subjects seem to lend support for the retinal model, this interpretation may be flawed if reach errors were to depend nonretinally on final eye position instead of being caused by an updated retinal representation. To examine this, in the following analysis, we further quantified reaching behavior by performing a multiple linear regression to investigate how the reach error relates to either eye displacement or final eye position. We fitted the following relationship, separately for the depth and direction dimension Err ⫽ a0 ⫹ a1 䡠 共Tret ⫺ u 䡠 ⌬E兲 ⫹ a2 䡠 Ef (1) to the data of each subject, with Err the reach error in degrees, Tret the retinal location (eccentricity or disparity) of the target, ⌬E the amount of eye displacement (version or vergence in °), Ef the final eye position in craniotopic coordinates (version or vergence in °), and a0, a1, u, and a2 free parameters in the fit. Parameter a0 quantifies the bias in the 99 • MAY 2008 • www.jn.org Downloaded from jn.physiology.org on August 11, 2008 that with those of the nonretinal model. Likewise we can ask the question of which reach pattern is observed when targets are presented with gaze near, but the reaches to them executed with gaze far. Also in this case, as shown in Fig. 4E, the reach pattern is more similar to that predicted by the retinal model (now being the static far pattern). To quantify these observations, we took the root of the average of the squared difference between the error in each dynamic trial and its predicted value by either the nonretinal or retinal model. Consistent with the qualitative observations, this yielded root-mean-squared error (RMSE) values lower for the retinal model than for the nonretinal model, for both dynamic conditions (0.8 vs. 2.2 and 0.7 vs. 2.0°, respectively). So for these pure vergence displacements, the subject’s average reach patterns seem to correspond best with the retinal coding scheme. The right panels of Fig. 4 present data of reaching movements across two directional gaze changes. Again, the two static conditions show different patterns of reach endpoints for different gaze directions (10° left and 10° right; Fig. 4B, left and right, respectively). Based on these results and using the same arguments described in the preceding text, our two models make two different predictions about the errors in the dynamic paradigm. Evidently, as shown by Fig. 4, D and F, which again are based on pure version trials (see METHODS), the observed reach patterns in the dynamic situations (left-to-right and right-to-left gaze changes, respectively) have a much greater similarity with the corresponding patterns predicted by the retinal model than with the endpoint distributions predicted by the nonretinal model. RMSE values, computed as in the preceding text, yielded 3.1 vs. 12.7 and 4.7 vs. 14.3° (retinal vs. nonretinal model) for the rightward and the leftward dynamic gaze changes, respectively. This corroborates findings in previous literature that also made a clear case for the retinal coding and updating of target direction (Henriques et al. 1998; Medendorp and Crawford 2002; Poljac and Van den Berg 2003). To further quantify the results of this subject (RV), we computed RMSE values related to the retinal and nonretinal model across all possible gaze displacement manipulations, thus also including the trials with a combined vergence-version change (see METHODS). In this analysis, performed in either dimension (depth/direction), we included in one manipulation the three reach patterns that were obtained with the eyes always starting at the same fixation point and ending at points that have the same vergence (or version) difference from this point, irrespective of the version (or vergence) component. The RMSE was then computed based on the average reach pattern from these three refixations (always 1 pure change and 2 combined version-vergence changes). Because we used nine initial fixation points, and two vergence (or version) differences relative to each point, this makes 18 manipulations in total per dimension. We plotted these values versus each other in Fig. 5, separately for the depth (A) and direction (B) dimension. Data points above the diagonal would indicate a preference for the nonretinal model; data below the diagonal would be more consistent with the retinal model. For this subject, this quantitative comparison clearly supported the retinal model, showing a better provision to account for the systematic depth and direction errors observed in the data. RMSE Retinal (°) 2286 DEPTH CODING FOR REACHING MOVEMENTS DEPTH Nonretinal 2287 DIRECTION Nonretinal Retinal B FP Far►Near FP Left►Right A Retinal RMSE=0.6° RMSE=1.2° RMSE=7.0° RMSE=1.7° RMSE=7.3° RMSE=2.0° D 14 N=14 subjects Direction B N=14 2.2 8 Final Eye Position 4 2 D 6 4 2 0 1.8 1.4 C 6 # Subjects RMSE Retinal (°) 2.6 Updating Gain 0 B Depth A DIRECTION A directional component, (P ⫽ 0.18; udirection ⫽ 1.13 ⫾ 0.36). The fact that the updating gain u is close to 1 indicates that the reach errors arise in relation to an updated retinal representation derived by correctly compensating for intervening eye displacements. To characterize the relative contribution of a retinally shifted eye-centered target representation and final eye position to the reach error, we computed the ratio (a1 ⫺ a2)/(a1 ⫹ a2). This ratio would be one when the reach error depends only on the updated target representation (a2 ⫽ 0) and would be minus one when the error depends solely on final eye position (a1 ⫽ 0). Figure 8, C and D, depicts this ratio separately for the depth and direction dimensions. As shown, for target depth, the ratio settles between the two extremes, showing a mean value of ⫺0.10 ⫾ 0.27, indicating that both final eye position and an updated disparity representation contributed about equally to 1.8 2.2 RMSE Nonretinal (°) 2.6 2 8 14 RMSE Nonretinal (°) FIG. 7. Comparison of the mean RMS error value (across all trials) for the retinal and nonretinal model for all subjects separately. In each subject, the retinal model performed best for both depth (A) and direction (B) updating as visualized by all respective data points lying below the diagonal. J Neurophysiol • VOL 0 1 -1 No Updating Perfect Updating Final Eye Position 0 1 Updated Target Position FIG. 8. Updating gains and final eye position effects in depth and directional reaching. A and B: updating gains (u) are distributed around a value of 1 for both depth and direction [u ⫽ 0.91 ⫾ 0.32 (SD) and u ⫽ 1.13 ⫾ 0.36, respectively]. C and D: contributions of the updated target position and final eye position to the reach error, expressed by the ratio (a1 ⫺ a2)/(a1 ⫹ a2). For depth, both contributed about equally, mean ratio: ⫺0.10 ⫾ 0.27. For direction, mean ratio: 0.65 ⫾ 0.39, meaning the error arises for the most part in relation to an updated direction representation. 99 • MAY 2008 • www.jn.org Downloaded from jn.physiology.org on August 11, 2008 reach error irrespective of target location or eye position. Parameters a1 and u characterize the error term related to the processing of an (updated) target representation relative to the eyes with a1 a scaling term and u the updating gain. If errors were to arise solely at the level of target presentation, the effect of eye displacement would be zero, thus the updating gain would be zero; u ⫽ 0. If the errors depend on the location of the target relative to the new eye position, this means that the system has taken possible eye displacements into account, which ideally requires the updating gain to be 1, thus u ⫽ 1. Finally, fit parameter a2 in Eq. 1 quantifies the dependence of the errors on final eye position per se. For all subjects, we found significant correlations: 0.2 ⬍ r ⬍ 0.9 (P ⬍ 0.05 for all subjects) for the depth dimension and 0.3 ⬍ r ⬍ 0.7 (P ⬍ 0.01 for all subjects) for the direction dimension. Parameter a0 had a mean value (⫾SD) that was significantly different from zero for depth (2.69 ⫾ 1.68, P ⬍ 0.001) but not for direction (⫺1.33 ⫾ 5.11, P ⫽ 0.35). The histograms in Fig. 8, left, show the distribution of updating gains u across subjects for both depth and directional updating. The data give no clear sign of differences between depth and directional updating. Across subjects the mean updating gain (mean ⫾ SD) is not significantly different from 1 either for the depth component (P ⫽ 0.33; udepth ⫽ 0.91 ⫾ 0.32) or for the # Subjects RMSE=0.3° RMSE=1.0° DEPTH FP Near►Far FP Right►left C FIG. 6. Population data. A and C: depth updating: reach patterns for 2 typical dynamic conditions (same as Fig. 4, in black), averaged across subjects, each superimposed on the predictions of the 2 models (in gray). Circle size, SE of each data point. The retinal model makes the best match to the data, for both conditions. B and D: direction updating: data in same format. Data are most consistent with predictions of the retinal model. 2288 S. VAN PELT AND W. P. MEDENDORP the reach errors. For the direction dimension, in contrast, the ratio had a value of 0.65 ⫾ 0.39, indicating that the reach errors seem to arise primarily in relation to an updated retinal representation. Binocular versus monocular updating DISCUSSION Over the last few decades, many studies have investigated how the constancy of spatial direction for motor actions is achieved across conjugate eye-movements. In contrast, the mechanisms involved in the maintenance of spatial depth across disjunctive eye movements have remained largely unexplored. Here we have addressed this issue using the accuracy of memory-guided reaching movements to visual targets briefly presented at different depths prior to a shift of gaze. We tested between two models of the implementation of depth constancy: a retinal versus a nonretinal model. To make this distinction, we exploited the fact that the accuracy of human reaching movements toward remembered space-fixed, but nonfoveally viewed, targets depends on the eyes’ fixation distance, analogous to the utilization of systematic reach errors for testing directional coding (Henriques et al. 1998). We hypothesized that if spatial depth is stored nonretinally, the intervening vergence shifts in the dynamic trials should have no effect on reaching. This model predicts that reaches in these trials should be similar to those made in static trials at the same initial eye position but without the intervening vergence shift. Alternatively, if depth coding is retinal, updating for the gaze shift becomes essential and reaches should match those of static trials performed at the final eye position under the assumption that the sensory consequences of the gaze shift have been perfectly taken into account (perfect updating). With intervening gaze-shifts, the memory-guided reaches showed an error pattern that was based on the new eye position and on the depth of the remembered target relative to that position (Figs. 3– 8). This suggests that target depth is recomputed after the gaze shift as would be required if the brain encoded depth in retinal coordinates. We found the values of J Neurophysiol • VOL 99 • MAY 2008 • www.jn.org Downloaded from jn.physiology.org on August 11, 2008 As a final note, throughout our analyses, we have assumed that target direction and depth are processed as separate signals. Theoretically it may be possible that depth information is not processed in the form of an explicit disparity signal but rather is computed on demand on the basis of two monocular direction representations that are stored and updated in separate maps. To investigate this possibility, we fitted Eq. 1 in terms of directional components only for each eye separately. On basis of the fitted parameters, we inferred reach depth by computing the point of intersection of the reach directions predicted based on monocular processing. We then quantified how well the resulting depth errors correlated with the actual, observed errors. Performance of this description in monocular coordinates was very poor, with 0.01 ⬍ r ⬍ 0.37 and significantly lower (paired t-test, P ⬍ 0.001 using Fisher z-transformation for comparing correlation coefficients) than predicted by the binocular coding scheme as shown in the preceding text. This warrants our assumption of binocular processing in terms of depth and direction components in the exploitation of the test described in the preceding text. the updating gain near unity (Fig. 8), demonstrating the persistence of a correct representation of target depth relative to fixation across vergence eye movements. This is in line with perceptual observations by Gonzalez at al. (1998), who reported that perceived depth of random-dot stereograms is not affected by changes in vergence. We deduce that the systematic reach errors in the present study must therefore arise after the updating stage in the subsequent visuomotor transformation from this updated retinal representation to the arm-centered representation for reaching (Khan et al. 2005). The latter is further emphasized by the influence of final eye position on the reach error (Fig. 8). As far as we are aware, no other studies have investigated the reference frame in depth constancy of across vergence eye movements. Krommenhoek and Van Gisbergen (1994), who tested human subjects in double-step eye movement experiments with combined version-vergence movements, reported that the saccadic and vergence system can use nonretinal feedback about a prior eye movement in direction and depth. They did not, however, address the spatial representation that underlies spatial constancy in this behavior. All subjects tested (n ⫽ 14) favored the retinal model of depth coding. This result is warranted by the fact that our control results on spatial direction, obtained in the same experiments, provide a strong confirmation of the earlier literature. Several behavioral studies on directional constancy have reported evidence for retinal updating of target direction (Henriques et al. 1998; Medendorp and Crawford 2002; Van Pelt and Medendorp 2007). Corroborating these findings, we found the updating gain for directional updating to be close to one (see Fig. 8). Moreover, several monkey and human brain areas show activity related to the retinal coding and updating of the direction of remembered targets, including frontal (Goldberg and Bruce 1990) and parietal areas (Batista et al. 1999; Duhamel et al. 1992; Medendorp et al. 2003a). The present study suggests that target depth is also coded and updated in retinal maps, presumably in the form of absolute disparity coordinates (Cumming and DeAngelis 2001). Changes of vergence alter the values of absolute disparities, so they must be updated to maintain spatial constancy. Previous neurophysiological work suggests that depth representations may be constructed in areas within occipital, frontal, and parietal cortex (Dobbins et al. 1998; Ferraina et al. 2000, 2002; Fukushima et al. 2002; Genovesio and Ferraina 2004; Genovesio et al. 2007; Gnadt and Beyer 1998; Gnadt and Mays 1995; Rosenbluth and Allman 2002; Sakata et al. 1997;). For example, Gnadt and Mays (1995) described neurons in the lateral intraparietal area (LIP) of the macaque that have threedimensional receptive fields. Activity of these neurons is expressed as a function of spatial parameters in the frontoparallel plane (horizontal and vertical eccentricity) and the relative depth from the plane of fixation (retinal disparity). Also Genovesio and Ferraina (2004) found LIP neurons that were sensitive to the retinal disparity of a target but further showed that this disparity tuning is modulated by fixation distance. A brief report from Bhattacharyya et al. (2005) on reaching in depth also suggest that neural activity in the parietal cortex reflects distance to target and vergence angle. Given these signals, it has been argued that the parietal cortex plays a role in the integration of retinal and extraretinal information to determine the egocentric distance of a target located in three- DEPTH CODING FOR REACHING MOVEMENTS J Neurophysiol • VOL two monocular signals is less consistent with our data, demonstrated by low correlations. That being said, in more complex updating conditions, depth and directional signals must interact to preserve spatial constancy in retinal coordinates (Ferraina et al. 2000; Li and Angelaki 2005; Medendorp et al. 2002, 2003b; Van Pelt and Medendorp 2007). For example, when the body translates, correct updating in a retinal frame requires updating to vary from object to object, depending nonlinearly on their depth and direction (Li and Angelaki 2005; Medendorp et al. 2003b). Recently we showed, using memory-guided reaching movements, that the updating of target direction for translational motion is compromised by small errors, which increase with depth from fixation and reverse in direction for opposite depths from fixation (Van Pelt and Medendorp 2007), consistent with translational updating in retinal coordinates. Li and Angelaki (2005) reported that monkeys can update target distance during body motion in depth using extraretinal vestibular information. Borne out by the present results, we propose that these vestibular signals interact with retinal disparity and eccentricity information to retain 3-D stability during body motion in space. ACKNOWLEDGMENTS The authors thank Dr. R. H. Cuijpers for valuable comments on an earlier version of the manuscript. GRANTS This work was supported by grants from the Netherlands Organization for Scientific Research (VIDI: 452-03-307) and the Human Frontier Science Program (CDA) to W. P. Medendorp. REFERENCES Andersen RA, Essick GK, Siegel RM. Encoding of spatial location by posterior parietal neurons. Science 230: 456 – 458, 1985. Baker JT, Harper TM, Snyder LH. Spatial memory following shifts of gaze. I. Saccades to memorized world-fixed and gaze-fixed targets. J Neurophysiol 89: 2564 –2576, 2003. Batista AP, Buneo CA, Snyder LH, Andersen RA. Reach plans in eyecentered coordinates. Science 285: 257–260, 1999. Baud-Bovy G, Viviani P. Pointing to kinesthetic targets in space. J Neurosci 18: 1528 –1545, 1998. Beurze SM, Van Pelt S, Medendorp WP. Behavioral reference frames for planning human reaching movements. J Neurophysiol 96: 352–362, 2006. Bhattacharyya R, Musallam S, Andersen RA. Reaching in depth: neural activity in posterior parietal cortex reflects distance to target and vergence angle. Soc Neurosci Abstr 289: 18, 2005. Bock O. Contribution of retinal versus extraretinal signals towards visual localization in goal-directed movements. Exp Brain Res 64: 476 – 482, 1986. Colby CL, Goldberg ME. 1999. Space and attention in the parietal cortex. Annu Rev Neurosci 22: 319 –349, 1999. Cumming BG, DeAngelis GC. The physiology of stereopsis. Annu Rev Neurosci 24: 203–238, 2001. DeAngelis GC. Seeing in three dimensions: the neurophysiology of stereopsis. TICS 4: 80 –90, 2000. Dobbins AC, Fiser J, Allman JM. Distance modulation of neural activity in the visual cortex. Science 281: 552–555, 1998. Duhamel JR, Colby CL, Goldberg ME. The updating of the representation of visual space in parietal cortex by intended eye movements. Science 255: 90 –92, 1992. Ferraina S, Paré M, Wurtz RH. Disparity sensitivity of frontal eye field neurons. J Neurophysiol 83: 625– 629, 2000. Ferraina S, Paré M, Wurtz RH. Comparison of cortico-cortical and corticocollicular signals for the generation of saccadic eye movements. J Neurophysiol 87: 845– 858, 2002. Flanders M, Tillery SI, Soechting JF. Early stages in a sensorimotor transformation. BBS 15: 309 –362, 1992. 99 • MAY 2008 • www.jn.org Downloaded from jn.physiology.org on August 11, 2008 dimensional (3-D) space (Genovesio and Ferraina 2004). It remains to be investigated whether the computation of egocentric depth is an automated process or is enforced on demand only when a (reach) action is prepared. Cumming and DeAngelis (2001) indicated that the updating of target distance may be expressed by changes in retinal disparity representations. Recently Genovesio et al. (2007) recorded neural activity in LIP while monkeys performed saccades between targets in different depths. They showed that in the postsaccadic period, neural activity is influenced conjunctively by both the eye displacement and the new eye position. In this respect, the behavioral observations made in the present study indicate a striking correspondence by showing effects of the same types of signals on the depth component of the reaching errors. It can be argued that these signals play a role in the dynamic retinal representation of visual space and in the further transformation of spatial information in other coordinates systems (Andersen et al. 1985; Genovesio et al. 2007). In support of this dynamic spatial representation, it has also been shown that neurons in LIP actually begin to respond before the eye movement to stimuli that will enter the receptive field after the eye movement (Duhamel et al. 1992; Nakamura and Colby 2002). In other words, LIP neurons anticipate the sensory consequences of the future eye position before the saccade is executed, which suggest that the updating mechanisms relies on a copy of the eye motor command (Sommer and Wurtz 2002) rather than on sensory feedback that arrives much later. A useful experiment to be performed in this context would be to investigate if predictive updating also occurs in relation to vergence eye movements. Along these lines, Kaiser and Lappe (2004) reported recently that visual objects flashed shortly before or during a saccade are mislocalized, resembling a compression of space around the saccade target. They attributed this distortion to the remapping process in parietal cortex, and it would be interesting to see whether similar spatial distortions occur across vergence eye movements, and if so, whether they have a similar time course. Viewed from a different perspective, a recent hypothesis here, put forward by Vaziri et al. (2006), is that the brain integrates the predicted sensory consequences of motor commands with the actual sensory (feedback) information to produce an estimate of sensory space that is better than possible from either source alone. More experiments are required to see if this hypothesis is upheld across combined saccade-vergence movements or in further extended conditions involving movements of the head and body in space. Finally, it is important to emphasize that the present study considered direction and depth as independent spatial variables, processed and updated separately during updating across eye movements. We made this assumption based on results of various reaching studies, performed in static conditions, showing that the spatial distributions of movement endpoints of reaches toward remembered targets were elliptical in shape with a tendency of the major axis to be directed to the subject’s eyes (Baud-Bovy and Viviani 1998; Henriques et al. 2003; McIntyre et al. 1997). This implies that noise in the reach is larger for the depth than for the directional component in these cases, suggesting that both dimensions are controlled separately. The present analyses supported this assumption by demonstrating that an alternative, implicit depth representation emerging from 2289 2290 S. VAN PELT AND W. P. MEDENDORP J Neurophysiol • VOL Medendorp WP, Smith MA, Tweed DB, Crawford JD. Rotational remapping in human spatial memory during eye and head motion. J Neurosci 22: 196RC, 2002. Medendorp WP, Tweed DB, Crawford JD. Motion parallax is computed in the updating of human spatial memory. J Neurosci 23: 8135– 8142, 2003b. Medendorp WP, Van Asselt S, Gielen CC. Pointing to remembered visual targets after active one-step self-displacements within reaching space. Exp Brain Res 125: 50 – 61, 1999. Mergner T, Nasios G, Maurer C, Becker W. Visual object localisation in space. Exp Brain Res 141: 33–52, 2001. Merriam EP, Genovese CR, Colby CL. Spatial updating in human parietal cortex. Neuron 39: 361–373, 2003. Merriam EP, Genovese CR, Colby CL. Remapping in human visual cortex. J Neurophysiol 97: 1738 –1755, 2007. Nakamura K, Colby CL. Updating of the visual representation in monkey striate and extrastriate cortex during saccades. Proc Natl Acad Sci USA 99: 4026 – 4031, 2002. Philbeck JW, Loomis JM. Comparison of two indicators of perceived egocentric distance under full-cue and reduced-cue conditions. JEP: Hum Perception Performance 23: 72– 85, 1997. Poljac E, Van den Berg AV. Localization of the plane of regard in space. Exp Brain Res 163: 457– 467, 2003. Rosenbluth D, Allman JM. The effect of gaze angle and fixation distance on the responses of neurons in V1, V2, and V4. Neuron 33: 143–149, 2002. Sakata H, Taira M, Kusunoki M, Murata A, Tanaka Y. The Trends Neurosci Lecture. The parietal association cortex in depth perception and visual control of hand action. Trends Neurosci 20: 350 –357, 1997. Schlag J, Schlag-Rey M, Dassonville P. Saccades can be aimed at the spatial location of targets flashed during pursuit. J Neurophysiol 64: 575–581, 1990. Shadmehr R, Wise SP. The Computational Neurobiology of Reaching and Pointing: A Foundation for Motor Learning. Cambridge, MA: MIT Press, 2005. Sommer MA, Wurtz RH. A pathway in primate brain for internal monitoring of movements. Science 296: 1480 –1482, 2002. Sparks DL, Mays LE. Spatial localization of saccade targets. I. Compensation for stimulation-induced perturbations in eye position. J Neurophysiol 49: 45– 63, 1983. Van Pelt S, Medendorp WP. Gaze-centered updating of remembered visual space during active whole-body translations. J Neurophysiol 97: 1209 – 1220, 2007. Van Pelt S, Van Gisbergen JA, Medendorp WP. Visuospatial memory computations during whole-body rotations in roll. J Neurophysiol 94: 1432–1442, 2005. Vaziri S, Diedrichsen J, Shadmehr R. Why does the brain predict sensory consequences of oculomotor commands? Optimal integration of the predicted and the actual sensory feedback. J Neurosci 26: 4188 – 4197, 2006. Vindras P, Desmurget M, Viviani P. Error parsing in visuomotor pointing reveals independent processing of amplitude and direction. J Neurophysiol 94: 1212–1224, 2005. Walker MF, Fitzgibbon J, Goldberg ME. Neurons of the monkey superior colliculus predict the visual result of impeding saccadic eye movements. J Neurophysiol 73: 1988 –2003, 1995. Wei M, DeAngelis GC, Angelaki DE. Do visual cues contribute to the neural estimate of viewing distance used by the oculomotor system? J Neurosci 23: 8340 – 8350, 2003. 99 • MAY 2008 • www.jn.org Downloaded from jn.physiology.org on August 11, 2008 Fukushima K, Yamanobe T, Shinmei Y, Fukushima J, Kurkin S, Peterson BW. Coding of smooth eye movements in three-dimensional space by frontal cortex. Nature 419: 157–162, 2002. Genovesio A, Brunamonti E, Giusti MA, Ferraina S. Postsaccadic activities in the posterior parietal cortex of primates are influenced by both eye movement vectors and eye position. J Neurosci 27: 3268 –3273, 2007. Genovesio A, Ferraina S. Integration of retinal disparity and fixation-distance related signals toward an egocentric coding of distance in the posterior parietal cortex of primates. J Neurophysiol 91: 2670 –2684, 2004. Gnadt JW, Beyer J. Eye movements in depth: what does the monkey’s parietal cortex tell the superior colliculus? Neuroreport 9: 233–238, 1998. Gnadt JW, Mays LE. Neurons in monkey parietal area LIP are tuned for eye-movement parameters in three-dimensional space. J Neurophysiol 73: 280 –297, 1995. Goldberg ME, Bruce CJ. Primate frontal eye fields. III. Maintenance of a spatially accurate saccade signal. J Neurophysiol 64: 489 –508, 1990. Gonzalez F, Rivadulla C, Perez R, Cadarso C. Depth perception in random dot stereograms is not affected by changes in either vergence or accommodation. Optom Vis Sci 75: 743–747, 1998. Hallett PE, Lightstone AD. Saccadic eye movements towards stimuli triggered by prior saccades. Vision Res 16: 99 –106, 1976. Henriques DY, Crawford JD. Direction-dependent distortions of retinocentric space in the visuomotor transformation for pointing. Exp Brain Res 132: 179 –194, 2000. Henriques DY, Klier EM, Smith MA, Lowy D, Crawford JD. Gazecentered remapping of remembered visual space in an open-loop pointing task. J Neurosci 18: 1583–1594, 1998. Henriques DY, Medendorp WP, Gielen CC, Crawford JD. Geometric computations underlying eye-hand coordination: orientations of the two eyes and the head. Exp Brain Res 152: 70 –78, 2003. Howard IP, Rogers BJ. Binocular Vision and Stereopsis. New York: Oxford Claridon, 1995. Israel I, Chapuis N, Glasauer S, Charade O, Berthoz A. Estimation of passive horizontal linear whole-body displacement in humans. J Neurophysiol 70: 1270 –1273, 1993. Julesz B. Foundations of Cyclopean Perception. Chicago: Chicago University Press, 1971. Khan AZ, Pisella L, Rossetti Y, Vighetto A, Crawford JD. Impairment of gaze-centered updating of reach targets in bilateral parietal-occipital damaged patients. Cereb Cortex 15: 1547–1560, 2005. Kaiser M, Lappe M. Perisaccadic mislocalization orthogonal to saccade direction. Neuron 41: 293–300, 2004. Klier EM, Angelaki DE, Hess BJM. Roles of gravitational cues and efference copy signals in the rotational updating of memory saccades. J Neurophysiol 94: 468 – 478, 2005. Krommenhoek KP, Van Gisbergen JA. Evidence for nonretinal feedback in combined version-vergence eye movements. Exp Brain Res 102: 95–109, 1994. Li N, Angelaki DE. Updating visual space during motion in depth. Neuron 48: 149 –158, 2005. McIntyre J, Stratta F, Lacquaniti F. A viewer-centered reference frame for pointing to memorized targets in three-dimensional space. J Neurophysiol 78: 1601–1618, 1997. Medendorp WP, Crawford JD. Visuospatial updating of reaching targets in near and far space. Neuroreport 13: 633– 636, 2002. Medendorp WP, Goltz HC, Vilis T, Crawford JD. Gaze-centered updating of visual space in human parietal cortex. J Neurosci 23: 6209 – 6214, 2003a.
* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project