Journal of Neurophysiology doi:10.1152/jn.01008.2009

Journal of Neurophysiology doi:10.1152/jn.01008.2009
Articles in PresS. J Neurophysiol (March 24, 2010). doi:10.1152/jn.01008.2009
Landmark stability important for ego/allocentric weighting
1
2
3
4
Cue reliability and a landmark stability heuristic determine relative weighting between
5
egocentric and allocentric visual information in memory-guided reach.
6
Patrick A. Byrne1,2 and J. Douglas Crawford1, 2,,3
7
8
1
9
2
10
11
12
Centre for Vision Research, York University, Toronto, Canada
3
Canadian Action and Perception Network
Neuroscience Graduate Diploma Program and Departments of Psychology, Biology, and
Kinesiology and Health Sciences, York University, Toronto, Canada
13
14
15
16
RUNNING HEAD: Landmark stability important for ego/allocentric weighting
17
18
19
20
21
22
23
24
25
26
27
28
Contact Info:
J. Douglas Crawford
Centre for Vision Research, York University
4700 Keele Street, Toronto, ON, Canada M3J 1P3
email: [email protected]
Tel: (416)736-2100 x88621
Fax: (416)736-5857
Copyright © 2010 by the American Physiological Society.
1
Landmark stability important for ego/allocentric weighting
2
29
30
31
ABSTRACT
It is not known how egocentric visual information (location of a target relative to the self) and
32
allocentric visual information (location of a target relative to external landmarks) are integrated
33
to form reach plans. Based on behavioural data from rodents and humans we hypothesized that
34
the degree of stability in visual landmarks would influence the relative weighting. Furthermore,
35
based on numerous cue-combination studies we hypothesized that the reach system would act
36
like a maximum-likelihood estimator (MLE), where the reliability of both cues determines their
37
relative weighting. To predict how these factors might interact we developed an MLE model that
38
weighs egocentric and allocentric information based on their respective reliabilities, and also on
39
an additional stability heuristic. We tested the predictions of this model in 10 human subjects by
40
manipulating landmark stability and reliability (via variable amplitude vibration of the landmarks
41
and variable amplitude gaze-shifts) in three reach-to-touch tasks: an egocentric control (reaching
42
without landmarks), an allocentric control (reaching relative to landmarks), and a cue-conflict
43
task (involving a subtle landmark ‘shift’ during the memory interval). Variability from all three
44
experiments was used to derive parameters for the MLE model, which was then used to simulate
45
egocentric-allocentric weighting in the cue-conflict experiment. As predicted by the model,
46
landmark vibration –despite its lack of influence on pointing variability (and hence allocentric
47
reliability) in the control experiment— had a strong influence on egocentric-allocentric
48
weighting. A reduced model without the stability heuristic was unable to reproduce this effect.
49
These results suggest heuristics for extrinsic cue stability are at least as important as reliability
50
for determining cue weighting in memory-guided reaching.
51
Keywords: memory-guided reaching, visuomotor transformation, egocentric, allocentric,
52
maximum-likelihood integration, human psychophysics
Landmark stability important for ego/allocentric weighting
3
53
54
55
INTRODUCTION
Goal directed actions –such as a reaching toward a briefly viewed target— often depend on feed-
56
forward movement plans, either because of the demands of movement speed (Carlton 1981;
57
Keele and Posner 1968; 1991; Zelaznik et al. 1983), because visual gaze is needed for some
58
other purpose (e.g. Flanagan et al. 2008), or because the target is no longer visible (Blohm and
59
Crawford 2007). The latter is often simulated in laboratory conditions, but it also occurs in
60
natural behaviors –such as hunting and gathering— where the object of interest frequently
61
becomes obscured, for instance by a bush. In these situations the brain must construct internal
62
spatial representations of target location and use these in a feed-forward fashion to guide the
63
movement (Ariff et al. 2002; Flanagan et al. 2001; Flanagan et al. 2003; Robinson 1981).
64
In theory there are two general ways to encode and remember the locations of visual
65
targets for action: relative to the self (egocentric coding) or relative to other external landmarks
66
(allocentric coding). For example, imagine a prehistoric hunter chasing his prey through the
67
savanna. Suddenly, his quarry disappears into tall grass. At this point the hunter has two ways to
68
aim a spear throw toward the hidden quarry. First, he might rely on egocentric information: the
69
(perhaps fading) memory of the last location at which the target was visible (i.e., where it
70
stimulated his retinas, taking into account where his eyes were pointing at the time).
71
Alternatively, he might rely on allocentric information: the memory of the animal’s last visible
72
location relative to some salient landmark (like a tuft of differently colored grass in his visual
73
field).
74
In real-world circumstances, both types of cue, egocentric and allocentric, are normally
75
available for the brain to use. Egocentric information is always present in healthy subjects, and
76
many studies have shown that subjects can reach and point with reasonable accuracy to
Landmark stability important for ego/allocentric weighting
4
77
remembered targets based solely on egocentric cues (Batista et al. 1999; Blohm and Crawford
78
2007; Buneo et al. 2002; Crawford et al. 2004). In most natural cases allocentric information can
79
also be derived from the environment, and it has been shown that this can have a strong influence
80
on remembered target location (Krigolson et al. 2007; Krigolson and Heath 2004; Obhi and
81
Goodale 2005). The question, then, is how are these cues combined and weighted by the brain?
82
Numerous studies have attempted to differentiate the factors that determine the relative
83
importance of these different cues. Diverse variables have been found to play an important role,
84
including age (Hanisch et al. 2001; Lemay et al. 2004), memory delay (e.g. Carrozzo et al. 2002;
85
Glover and Dixon 2004; Hay and Redon 2006; Obhi and Goodale 2005), context (Neely et al.
86
2008), and demand characteristics (Bridgeman et al. 1997). Allocentric information can also
87
affect reaching movements differentially depending on the relative alignment between effector
88
movement direction and intrinsic landmark geometry (de Grave et al. 2004). Furthermore, it has
89
been suggested by many that allocentric information tends to dominate over egocentric when the
90
former is present, at least when action occurs after a memory delay (e.g. Lemay et al. 2004;
91
Neggers et al. 2005; Sheth and Shimojo 2004). However, to our knowledge, the computational
92
rules used to weight between such cues have not been tested or modeled.
93
One factor that is likely to influence the weighting of egocentric and allocentric
94
information is the relative reliability of these two sources of information. In practice, the
95
reliability of cue is taken to be the inverse of the variance in repeated behavioral responses based
96
solely on that cue (for a recent experimental example, see Brouwer and Knill 2009). As a ‘real
97
world’ example, when our hunter bases his spear throws on distal landmarks, he might find that
98
he has more difficulty hitting his target (the endpoint of his spear toss might be more variable
99
over repeated throws) than if he relied on nearby landmarks. In the former case, when only distal
Landmark stability important for ego/allocentric weighting
5
100
landmarks are available, he might tend to give more weight to his own egocentric memory of
101
target location than he would in the latter case in order to compensate. From numerous
102
experimental studies requiring subjects to respond based upon two or more estimates of a given
103
stimulus dimension, it has been found that the relative influence of these multiple cues is, at least
104
in part, determined by their respective reliabilities, as measured from response variability in
105
single cue control tasks (e.g. Battaglia et al. 2003; Ernst and Banks 2002; Knill 2007b; Knill and
106
Saunders 2003; van der Kamp et al. 1997; Vaziri et al. 2006). Thus, we expect any putative
107
combination rule for egocentric and allocentric spatial cues to show similar dependence.
108
Although in many cases the brain does appear to combine multiple information sources
109
based on accurate estimates of individual cue reliabilities, this need not always be the case. The
110
brain might also derive heuristic rules for judging cue reliability through prior experience or
111
evolutionary hardwiring. Returning again to our hunter story, if a strong wind was causing the
112
landmark (the colored tuft of grass) to wave back and forth and change shape, the hunter’s brain
113
might discount this landmark as unreliable, even though its average position in fact remains
114
rooted in the same location. This might be because in previous cases his visual system noticed
115
that loose vegetation blowing in the wind has no value as an allocentric cue, and thus has learned
116
to place less trust on anything in motion. We refer to such putative down-weighting of allocentric
117
information as a ‘stability heuristic’, that, if it exists, likely results from expectations about the
118
usefulness of landmarks. Presumably, spatial information derived from apparently stable
119
landmarks would weigh more heavily in an egocentric-allocentric combination than would
120
information derived from apparently unstable ones. This question has been addressed in several
121
studies of spatial cognition (Biegler and Morris 1996a; Biegler and Morris 1993, 1996b; Burgess
122
et al. 2004; Jeffery 1998). For example, place-cells in the rat may cease to fire for landmarks that
Landmark stability important for ego/allocentric weighting
123
are shifted in the presence of the animal (Jeffery 1998). Likewise, rats will only learn the
124
locations of food rewards relative to landmarks if those landmarks are stable Biegler and Morris
125
(1996a, 1993, 1996b). Similarly, humans perform better in spatial memory tasks when visual
126
landmarks never change location in the presence of the subject (Burgess et al. 2004). However,
127
to our knowledge, the behavioural consequences of variable apparent landmark stability on cue-
128
combination have not been investigated directly and quantitatively in any studies of human
129
visuomotor control.
130
6
In order to simultaneously test the influence of these factors (actual egocentric and
131
allocentric reliabilities, and heuristically-based judgements of landmark reliability) it is
132
necessary to make quantitative predictions. This is not trivial. For example, introducing
133
instability in landmarks might affect both the actual reliability of allocentric information (as
134
judged from response variability in a task where only allocentric information can be used) and
135
activate the putative stability heuristic. These factors, along with estimates of egocentric
136
reliability might then interact in very complex ways, especially when one is dealing with a two-
137
dimensional array of targets. Previous studies of both perception and action have dealt with such
138
problems by using a maximum-likelihood estimator (MLE) (e.g. Battaglia et al. 2003; Ernst and
139
Banks 2002); (Knill 2007b; Knill and Saunders 2003; van der Kamp et al. 1997; Vaziri et al.
140
2006). An MLE model allows one to predict how multiple stimulus estimates with different
141
reliabilities should combine in a statistically optimal fashion, which is exactly what we needed to
142
do here.
143
In the current study we directly tested the hypotheses that 1) reaching to remembered
144
targets is guided by an internal weighting process that combines egocentric and allocentric
145
information, 2) that allocentric information derived from apparently unstable landmarks is
Landmark stability important for ego/allocentric weighting
146
weighed less than that derived from apparently stable ones (because of a stability heuristic), and
147
that 3) this weighting process is also reliability-dependent. We did this by first developing an
148
MLE model of reaching that relied on both cue reliability and the stability heuristic. Within the
149
model, the stability heuristic was represented via a ‘stability parameter’ that affected weighting
150
of egocentric and allocentric information by modulating the influence of the actual reliability of
151
the latter. Second, we experimentally derived the parameters of this model. Finally, we used the
152
fitted model to simulate and predict the results of a reach-to-touch paradigm in which a spatial
153
conflict between egocentrically and allocentrically defined target locations was induced, and in
154
which the stability of visual landmarks and the actual reliability of egocentric information were
155
systematically varied. As predicted by our model and confirmed in the results, both the stability
156
heuristic and the actual reliability of egocentric and allocentric information contributed to the
157
relative weighting of these cues.
158
159
160
161
162
163
164
METHODS
Theory and Design
An MLE rule weighs multiple estimates in proportion to their reliabilities. Such rules
165
have been found to act across modalities (e.g. Battaglia et al. 2003; Ernst and Banks 2002) and
166
within the visual modality alone (Knill 2007b; Knill and Saunders 2003; van der Kamp et al.
167
1997; Vaziri et al. 2006). If landmark stability influences egocentric-allocentric weighting, it
168
might do so in at least two ways. First, as suggested by the experiments of Burgess et al. (2004)
169
(and others described above), there might be some internal stability heuristic that causes the
170
brain to down-weigh the contribution of allocentric information based on landmarks that do not
171
appear to be fixed at a particular location. This effect would be independent of the actual
7
Landmark stability important for ego/allocentric weighting
172
reliability of the allocentric information. Second, it might be more difficult to localize a target
173
relative to landmarks that undergo any kind of movement, even if the movement could, in
174
principle, be averaged out. Down-weighting of allocentric information in this latter case would
175
be reliability-dependent.
176
8
Since we framed our hypotheses in terms of an MLE model, it was necessary to develop
177
this model in concert with the experimental design such that 1) some aspects of the data could be
178
used to fit the model parameters, whereas 2) other aspects of the data could be used to test the
179
model (importantly, while maintaining mutual independence between these two procedures). In
180
brief, there were three tasks in which subjects reached to touch the remembered location of a
181
visual target flashed briefly on a computer screen in complete darkness, after a memory delay.
182
These tasks consisted of a cue-conflict experiment (Figure 1A) in which egocentric and
183
allocentric cues conflicted at test because of a subtle landmark shift during the memory delay,
184
and two controls: an egocentric-variability control (Figure 1B) designed to measure reaching
185
variability when no landmarks were present, and an allocentric-variability control (Figure 1C)
186
designed to measure reaching variability when reaching could depend only on visual landmarks.
187
Visual landmarks were chosen to be similar to those of Krigolson and Heath (2004), which have
188
been shown to generate significant improvement in reaching accuracy to remembered targets.
189
To this basic design we added the following manipulations. First, we manipulated
190
stability of landmarks (in both the cue-conflict experiment and the allocentric control) by
191
imparting a vibration to them. The main intent of this manipulation was to confirm the existence
192
of the stability heuristic in egocentric-allocentric weighting, but it was also possible that this
193
manipulation would affect the actual reliability of allocentric information. Our allocentric control
194
experiment allowed us to measure the latter via response variability and incorporate this into the
Landmark stability important for ego/allocentric weighting
9
195
MLE model. Second, we added another manipulation to produce corresponding variations in the
196
reliability of the egocentric channel. It has been shown that efference copies of eye position and
197
eye movement are used to update and maintain spatial representations within the brain (e.g.
198
Niemeier et al. 2003), and that increasing the amplitude of gaze shifts that occur during a
199
memory interval increases the amount of noise in spatial memory (Prime et al. 2006; Prime et al.
200
2007). Thus, during the memory delay we manipulated egocentric reliability by varying total
201
gaze movement amplitude during the memory interval. Our egocentric control allowed us to
202
independently measure the effects of this manipulation and incorporate this into our model.
203
We modeled the stability heuristic by adding a ‘stability parameter’ to our MLE model
204
(see below for model details) that artificially deflates the reliability estimate for allocentric
205
information when landmarks are unstable. This introduced the problem of how to determine a
206
value for this parameter. Normally, testing an MLE model of cue-combination involves
207
measuring cue reliability from response variability in single-cue control tasks. From these
208
reliability estimates, the MLE model can be used to predict how subjects will weigh the various
209
cues when these are simultaneously present, but possibly in conflict (e.g. Smeets et al. 2006; van
210
Beers et al. 1999). In order to obtain estimates for egocentric and allocentric reliability, of motor
211
noise, and an estimate for the stability parameter, we instead followed procedures similar to
212
those of Brouwer and Knill (2009). These authors noted that MLE models predict a specific
213
relationship between response variability in single-cue control tasks and variability in a
214
corresponding multi-cue task. In our case the stability parameter also entered into this
215
relationship. Therefore, we could use this relationship between reaching variability in all three of
216
our tasks to obtain estimates of egocentric and allocentric reliability, of motor noise, and of the
217
stability parameter. These estimates were then incorporated into our MLE model and used to
Landmark stability important for ego/allocentric weighting
10
218
predict what the weighting should be between egocentric and allocentric cues in our cue-conflict
219
experiment (see Figure 2 for a graphical illustration of our procedure). Note that our procedure
220
for determining the value of our stability parameter is not novel; for example, McGuire and
221
Sabes (2009) used a similar approach to determine values for non-reliability related parameters
222
in their MLE model.
223
The mathematical details of our MLE model are presented below (and in the Appendix)
224
after a description of the experimental procedures used to obtain the reaching dataset.
225
226
227
228
Participants
229
four males between the ages of 20 and 49. Nine of the ten subjects were naïve to the design and
230
purpose of the experiment, while one was naïve only to the design. This latter subject showed
231
results that did not differ qualitatively from the remaining subjects. All subjects had normal or
232
corrected to normal vision and none of these subjects had any known neuromuscular deficits. All
233
subjects gave informed consent and all procedures involved in the experiment were approved by
234
the York Human Participants Review Subcommittee.
235
236
237
238
Apparatus and Stimuli
239
personalized dental impression. The heights of the seat and bite bar were adjusted independently
240
so that the nasal root was vertically and horizontally aligned with the centre of a CRT display
241
(Dell). The screen had vertical and horizontal screen dimensions of 30 cm (1024 pixels) and 40.5
242
cm (1280 pixels), a refresh rate of 70 Hz, and was situated 40 cm directly in front of the subject.
243
In order to eliminate background luminance (stimuli were presented on a black background in a
A total of ten right-handed human subjects participated in all three experiments; six females and
Subjects were seated in total darkness with the head fixed using a bite bar apparatus with a
Landmark stability important for ego/allocentric weighting
11
244
completely dark room) the CRT brightness was set to the minimum setting and a light absorbing
245
film was applied to the screen surface. All stimuli were displayed on this screen, with the
246
exception of a beep that indicated when subjects were to reach. Two 40 Watt desk lamps, one
247
placed on either side of the CRT display, were also turned on automatically at regular intervals
248
(see below) in order to eliminate dark adaptation. Between trials the subject was instructed to
249
return their fingertip to a home location positioned near the bottom right corner of the CRT on
250
the table that supported it. At this location a coin was glued to the table to provide a distinctive
251
surface. With their fingertip at the home location the subject’s arm was resting comfortably on a
252
table at the same height as the base of the CRT display.
253
Reaching responses were measured using a two camera Optotrak 3020 (Northern Digital)
254
tracking system. These cameras continuously recorded (sampling frequency of 150 Hz) the 3-D
255
positions of three infrared-emitting diodes (IREDs) placed along the right index finger, with one
256
near the fingertip, another approximately 1 cm more proximal along the finger, and another
257
approximately 1 cm further proximal. IRED position data from the Optotrak was not filtered.
258
Gaze-direction was continuously monitored (sampling frequency of 120 Hz) by a head-mounted
259
infrared eye-tracking system (Applied Science Laboratories) that monitored the left eye. Eye-
260
tracking data was filtered to remove rapid signal changes corresponding to unnatural eye
261
movement speeds of greater than 1000 deg/s. This was accomplished simply by removing the
262
data starting at the high speed movement onset and the point of return to pre-movement baseline.
263
The empty space was interpolated if it did not last more than 400 ms, otherwise the trial was
264
discarded. The same interpolation procedure was used to remove eyeblinks.
265
266
All stimuli were generated with a Windows-based Pentium 4 PC (Dell) using MATLAB
6.5 (The MathWorks) along with the Psychophysical ToolBox v3.0.8 (Brainard 1997; Pelli
Landmark stability important for ego/allocentric weighting
12
267
1997). The to-be-remembered target stimulus consisted of a single, filled yellow disc with a
268
diameter of one degree visual angle. For a given trial, this target stimulus could appear
269
anywhere on a circular annulus with inner radius of 11 degrees and outer radius of 13 degrees
270
centered at the screen center. The visual landmarks consisted of four identical blue discs, each
271
with a diameter of one degree, positioned at the vertices of a virtual square with a seven degree
272
edge length. On any given trial this virtual square was positioned so that the to-be-remembered
273
target occupied a random location within a smaller central square region of 60% of the width of
274
the full virtual square. Furthermore, the virtual square, and hence the collection of visual
275
landmarks, would vibrate about its average position with either a small or large amplitude. In
276
particular, the small vibration amplitude was chosen so that each individual landmark in this
277
condition maintained a relatively large region of overlap with its initial position at all times. Thus,
278
landmarks in this condition were taken as ‘stable’ because they appeared to wobble about in
279
place but not to change location completely. We chose the large vibration amplitude such that
280
each individual landmark maintained no constant region of overlap with itself and thus appeared
281
to move from place to place within a limited region of space. In both low vibration amplitude
282
(stable) and high vibration amplitude (unstable) conditions the landmarks had well-defined
283
average locations (i.e. there was no net ‘drift’) and could, in principle, have been equally useful
284
to subjects. Our assumption here was that the relatively unstable, larger vibration amplitude
285
landmarks would be judged as less useful by subjects as an allocentric cue. The vibration itself
286
consisted of independent horizontal and vertical sinusoidal motion, with horizontal and vertical
287
oscillation frequencies on each trial being chosen randomly and independently from the range
288
[6.67,10] Hz. Choosing both vibration frequency components independently ensured that the
289
overall motion did not appear to be circular. The small and large vibration amplitudes were
Landmark stability important for ego/allocentric weighting
290
chosen to be 0.2 and 0.6 degrees, respectively, which satisfied the definitions of stable and
291
unstable given above.
13
292
Visual fixation during the experiment was controlled by means of a fixation cross
293
consisting of two identical bars that had a width of 0.17 degrees and a length of 0.67 degrees. At
294
the beginning of each trial and throughout target presentation the fixation cross would be present
295
at the center of the screen. Gaze shifts of either small or large amplitude were generated by
296
having the fixation cross make a sequence of two jumps. During such a sequence, the cross
297
would first disappear from the screen center and then reappear at an intermediate location 100
298
ms later. After 750 ms the cross would disappear again and reappear at its final location 100 ms
299
later. The intermediate and final locations were chosen randomly within two constraints: 1) the
300
final location had to be within a disc of six degrees radius centered at the original location, and 2)
301
the overall movement amplitude had to be either small (10 degrees) or large (30 degrees).
302
Throughout this sequence subjects were required to follow the cross with their eyes.
303
304
305
306
Experimental paradigm
307
experiment, the second session was the egocentric-variability control experiment, while the third
308
was the allocentric-variability control experiment. All sessions were performed on separate days
309
separated by two weeks or more. Each session also began with a simple calibration block that
310
allowed IRED positions to be converted easily into screen-relative reach endpoint coordinates.
311
The two control experiments were run after the cue-conflict experiment in order to ensure that
312
they did not somehow affect the behaviour of subjects in the main experiment.
313
The entire experiment consisted of three sessions, the first of which was the main cue-conflict
Landmark stability important for ego/allocentric weighting
14
314
315
316
Cue-conflict experiment
317
centrally-presented cross for 2 s. At the end of this period the yellow target disc and vibrating
318
visual landmarks would appear for 1.5 s, with the target situated randomly within the centered
319
annulus and the landmarks situated relative to the target as described above. Although the visual
320
landmarks would vibrate whenever present, the target itself was always perfectly stationary
321
whenever visible. Trials with small or large vibration amplitude, i.e. stable or unstable trials,
322
were randomly interleaved. Furthermore, subjects were explicitly instructed to ignore the
323
“vibrating blue dots”. Following a 500 ms delay after target/landmark offset the fixation cross
324
would execute the small or large movement sequence described above, with small and large
325
movement trials randomly interleaved. In total then, we had four unique experimental conditions,
326
ν . These were sv_sgs (small landmark vibration-small gaze shift), sv_lgs (small landmark
327
vibration-large gaze shift), lv_sgs (large landmark vibration-small gaze shift), or lv_lgs (large
328
landmark vibration-large gaze shift). In order to introduce cue-conflict the vibrating landmarks
329
would reappear 300 ms after completion of the eye-movement sequence for another 1.5 s, but
330
with their collective center shifted in a random direction by three visual degrees. The rationale
331
here was that the shift in landmarks should have had no effect on an egocentric memory of target
332
location, but that reaching based upon allocentric information would be shifted with the
333
landmarks. Thus, the location that the target would occupy if it had shifted with the visual
334
landmarks will be referred to from here on as the allocentric location. At landmark offset the
335
fixation cross would disappear and the subject would hear a beep indicating that they should
336
touch the screen at the remembered location of the yellow target disc. Subjects were allowed to
337
direct their gaze freely throughout the reaching phase. After another 2.5 s a second, return-signal
Each trial of the cue-conflict experiment (see Figure 1A) began with the subject fixating the
Landmark stability important for ego/allocentric weighting
15
338
beep would sound indicating that the subject should return their finger to the home position. The
339
next trial would begin immediately. Every fifth trial was a “throwaway” trial during which the
340
40 Watt lamps were illuminated to prevent dark adaptation. After every 20 trials subjects were
341
given a 35 s rest period during which the lamps were illuminated. In total subjects performed
342
130 non-illuminated trials, with results from the first ten discarded as practice trials. Sample eye
343
and finger traces for one subject are shown in Supplementary Figure S1.
344
345
346
347
Egocentric-variability control experiment
348
control experiment was identical in all aspects to the main experiment described above, with the
349
exception that no landmarks were ever presented. Thus, subjects presumably only ever had
350
egocentric information about target location to work with. Again, every fifth trial was a
351
“throwaway” trial during which the 40 Watt lamps were illuminated to prevent dark adaptation.
352
After every 20 trials subjects were given a 35 s rest period during which the lamps were
353
illuminated. In total subjects performed 90 non-illuminated trials, with results from the first ten
354
discarded as practice trials.
355
356
357
358
Allocentric-variability control experiment
359
main experiment. There were three differences. First, subjects were instructed explicitly to
360
remember where the target was relative to the “vibrating blue dots”. Second, the small shift in
361
location of the vibrating visual landmarks from first to second presentation was accompanied by
362
an additional, large translational shift. In order to generate this shift, the vector connecting the
363
subject’s nasal root and the original landmark centre of geometry was rotated about the axis
The egocentric-variability control experiment is depicted in Figure 1B. The procedure for this
The allocentric-variability control experiment is shown in Figure 1C was nearly identical to the
Landmark stability important for ego/allocentric weighting
16
364
connecting the subject’s nasal root and the screen centre by a random angle of between 45 and
365
315 degrees. The tip of this rotated vector was taken as the new centre of geometry for the
366
translated landmarks. Thus, the location of the landmarks on their second presentation was
367
unrelated to their location during the first presentation, but was subject to the same overall
368
constraints on possible location. Third, during the reaching phase of this task, subjects were
369
required to touch to the location that the target would have had if it had shifted with the
370
landmarks.
371
372
373
374
Optotrak calibration
375
to touch a yellow target disc that would appear at a random location within the centered annulus
376
described above. IRED position data in the Optotrak intrinsic coordinate system was then
377
combined offline with the known screen coordinates for the various target presentations to
378
generate a linear mapping between IRED position and screen coordinates. This procedure
379
eliminated the need to place precisely the CRT screen relative to the Optotrak coordinate system,
380
and it eliminated the need to place precisely and identically the IREDs on the fingers of different
381
subjects.
This calibration session consisted of 20 simple trials in which the head-fixed subject would reach
382
383
384
385
Data Analysis
386
Each trial from all experimental sessions (calibration, egocentric-variability control, allocentric-
387
variability control, and main cue-conflict experiment) involved a reaching response phase that
388
began with a beep signaling the start of reaching, a 2.5 s movement period, and a second, return-
389
signal beep indicating that the subject was to return their finger to the home location. Any trial in
All data analysis occurred offline using custom software written in MATLAB (The MathWorks).
Landmark stability important for ego/allocentric weighting
17
390
which the subject’s finger moved faster than 1 cm/s before the start signal was discarded. In
391
order to determine IRED coordinates for a given reaching response the Optotrak-measured IRED
392
positions were averaged over an approximately 300 ms period that occurred within the
393
movement period and in close temporal proximity to the corresponding return-signal beep. For
394
most trials this averaging period began 300 ms before the return signal and ended at the return
395
signal. However, sometimes a subject would begin returning their fingertip early or would make
396
a finger movement exceeding a criterion velocity of five cm/s within this time period. In such
397
cases the last 300 ms period preceding the return signal in which the velocity criterion was not
398
exceeded was selected as the averaging period. This was done, as opposed to choosing one
399
decelaration point (as just one on many examples, see Krigolson et al. 2007), in order to ensure
400
subjects had reached their final selected position and to smooth out irrelevant noise (e.g. van
401
Beers et al. 1999).
402
In order to generate a mapping between IRED coordinates and screen-relative
403
coordinates of the fingertip at the end of a reaching movement the known screen coordinates of
404
the target presentations in the calibration sessions were regressed against IRED coordinates
405
determined using the averaging procedure described above. This regression was a simple least-
406
squares fitting of an eight parameter linear model. Once the calibration parameters were
407
determined IRED coordinates of a reaching endpoint could be mapped to screen-relative
408
coordinates for the control and main experiment sessions. The fitting procedure was carried out
409
independently for each of the three IREDs, and the IRED that generated the fit with the smallest
410
Predicted Residual Sum of Squares (Allen 1974) statistic was used to determine screen-relative
411
reaching endpoints in the subsequent control or main experiment session. This measure is more
Landmark stability important for ego/allocentric weighting
18
412
appropriate than a simple R^2 value because it measures the predictive ability of the fit – exactly
413
what we wish to know.
414
During each trial of the control or main experiments fixation was deemed acceptable if
415
gaze did not deviate from the cross by more than +/-1 degree in the horizontal or vertical
416
direction. The gaze shift sequence was deemed acceptable if: 1) the first eye movement began
417
after the cross appeared at the intermediate location and reached the cross at this location before
418
it disappeared, and 2) the second eye movement began after the cross appeared at its final
419
location and reached the cross at this location within 300 ms of its appearance (thus ensuring
420
gaze was properly located at the start of the second landmark presentation phase). Any trial in
421
which gaze shifts did not satisfy these criteria was discarded. Furthermore, the raw data was
422
trimmed of outliers using the Chauvenet procedure (Taylor 1997). For both control experiments
423
the correct target location for a given trial was subtracted from the raw reaching endpoint to
424
generate a set of target-relative responses. The Chauvenet procedure was applied to both the x
425
and y components of the set of target-relative reaching endpoints for each subject/condition
426
conjunction. In the main cue-conflict experiment the model in Equation 1 was fit to the raw data
427
for each subject/condition conjunction and the Chauvenet procedure was applied to the x and y
428
components of the fit residuals.
429
In the main cue-conflict experiment target location, landmark shift direction, and
430
landmark position relative to target were chosen random within the previously described
431
constraints. In order to make data from each trial comparable the transformation process
432
depicted in Figure 3A was performed on the reaching data. First, reaching endpoints for a given
433
subject were corrected to remove any systematic reaching bias (not depicted in the Figure). That
434
is, the vector connecting target and reaching endpoint for a given trial was averaged across all
Landmark stability important for ego/allocentric weighting
19
435
conditions and trials for a given subject and was subtracted from all of that subject’s individual
436
reaching endpoints. Next, the corrected reaching endpoint from a given trial was transformed by
437
the unique set of translation, rotation, and scaling operations that would bring the original target
438
location and allocentric location, if similarly transformed, to the origin and the (1,0) position of
439
the new coordinate system, respectively. The x-component of this transformed reaching
440
endpoint will be referred to as its allocentric weight. Thus, an allocentric weight of 0 would
441
imply a reaching endpoint at the original target location (neglecting the component perpendicular
442
to the shift direction) and an allocentric weight of 1 would imply a reaching endpoint at the
443
allocentric location (the location of the target if it had shifted with the landmarks). Note: when
444
we fit our MLE model below, we fit it to the raw data without any of the above transformations.
445
For the control experiments a measure of overall variable reaching error relative to target
446
location was required for each combination of subject and eye-movement/vibration amplitude
447
condition (small/large). Each reaching endpoint (in screen coordinates) for a given combination
448
was translated by subtracting from it the actual target location on that trial. The overall reaching
449
variance estimate of these target-relative responses was taken to be the root-mean-square of the
450
eigenvalues of their covariance matrix. We chose this measure because it behaves like the area of
451
a confidence ellipse for relatively isotropic reaching endpoint distributions, but more like a one-
452
dimensional variance for highly elongated distributions. In the latter case one could in principle
453
find a confidence ellipse that was very long in one dimension, but short enough in the other that
454
it maintained a relatively small area. In this case area would not be a particularly good measure
455
of reaching variability.
456
457
Details of Model.
Landmark stability important for ego/allocentric weighting
20
458
When reaching for a target we assumed that the brain relies on at least two estimates of target
459
location, rˆa;ν ,s and rˆe;ν , s , where rˆa;ν ,s is an allocentric estimate based on visual landmarks and
460
rˆe;ν , s is an egocentric estimate. Here, ν refers to the collection of conditions under which the
461
target is perceived and the reaching takes place, while s refers to the fact that these estimates
462
might vary systematically between individual subjects, even under otherwise identical conditions.
463
We further assumed that these estimates are bivariate normal random variables (we restrict the
464
model to two spatial dimensions here) with expectation values of ra and re , the actual
465
allocentrically and egocentrically-defined target locations, and covariances of Σa;ν , s and Σe;ν , s .
466
The overall form of our model was chosen to be a simple, but general linear mapping of the form
467
{
(
)
}
rp = Mν , s Wν , s rˆeν , s + I − Wν , s rˆaν , s + bν , s + ε m
s
,
(1)
468
where rp is the subject’s reaching endpoint, Wν , s is a two dimensional weight matrix, and ε ms is
469
additive bivariate normal motor noise. Here we have assumed that the egocentric-allocentric
470
integrator is unbiased, but that the result may be affected by small, systematic distortions, which
471
we modeled linearly via the multiplicative matrix, Mν ,s , and the constant offset vector, bν ,s . A
472
similar modeling scheme has been used successfully elsewhere (Brouwer and Knill 2009) for a
473
one-dimensional task. For simplicity, we can rewrite Equation 1 as
474
rp = Mν , s Wν , s re + I − Wν , s ra + bν , s + εν , s
475
where εν ,s is a new bivariate normal random variable, which depends on Mν ,s , Wν ,s , Σa;ν ,s ,
476
Σe;ν ,s , and ε m
s , and contains all response variability present in Equation 1.
477
478
{
(
) }
,
(2)
Next, if an individual subject were actually performing a reliability-dependent MLE
integration of stored egocentric and allocentric information about target location, then the weight
Landmark stability important for ego/allocentric weighting
21
479
matrices in Equation 1 (and therefore 2) should be determined fully and uniquely by the
480
variability inherent in estimates derived from these cues. In direct analogy with the well-known
481
one dimensional case, the weighting of each cue would be given by
482
−1
−1
Wν , s = pν−,1s Σ a;
ν , s + Σ e;ν , s
483
where the various symbols have the same meaning as above and pν ,s = 1 . Finally, we assumed
484
that landmark instability would affect the weight matrices in Equation 3 independently of the
485
actual reliability of egocentric or allocentric information. We modeled this with the addition of
486
the stability parameter, pν ,s , which was intended to represent the effects of a stability heuristic
487
that modulates the apparent reliability of allocentric information based upon unstable landmarks.
488
More specifically, if this parameter is greater than one, then it has the effect of making
489
allocentric information look less reliable (more variable) in Equation 3 . Thus, for stable
490
landmarks we take pν ,s = 1 , while for unstable landmarks we have pν ,s > 1 . This approach required
491
fewer parameters than a fully Bayesian model with a prior.
(
)
−1
−1
Σe;
ν ,s ,
(3)
492
If we wish to test the reliability-dependent MLE model defined by Equation 1 &
493
Equation 3, then we need to find an estimate of pν ,s . This was accomplished by assuming that
494
subjects were performing an MLE combination, and then using the predicted relationship
495
between variability in the three experiments to determine uniquely the values for Σa;ν ,s , Σe;ν ,s ,
496
the motor noise covariance matrix and pν ,s . If the MLE assumption were in fact correct, then
497
mean reaching endpoints in the cue-conflict experiment should be well-described by the
498
combination of Equation 1 and Equation 3.
499
500
501
Model Fitting
Landmark stability important for ego/allocentric weighting
22
502
After standard least-squares fitting to the raw endpoint dataset (no removal of systematic biases
503
or transformation to allocentric weights, etc.), we used Equation 2 without the combined
504
variability term to generate a set of predicted reaching endpoints for each subject in each
505
condition. These calculated values were then transformed (as described above) into a
506
corresponding set of allocentric weights, which we refer to here as direct-fit allocentric weights.
507
Next, after determining estimates for Σa;ν ,s , Σe;ν ,s , the motor noise covariance matrix, and the
508
stability parameter, pν ,s from reaching endpoint variability (details in the Appendix), we
509
replaced the directly fitted values of Wν ,s in Equation 2 with the values from Equation 3. This
510
allowed us to calculate the set of MLE allocentric weights for each subject in each experimental
511
condition. If subjects really were performing an MLE combination of egocentric and allocentric
512
information in the cue-conflict experiment, as we assumed, then the MLE allocentric weights
513
should be identical to the direct-fit allocentric weights. In order to compare the between-subjects
514
means for MLE and direct-fit allocentric weights in each experimental condition we performed a
515
Bootstrapping procedure (see Appendix).
516
517
518
519
520
521
RESULTS
General Effect of landmark shift
Before examining the effects of reliability or the stability heuristic on cue-combination we first
522
confirmed that both egocentric and allocentric information were being combined by subjects in
523
the main cue-conflict experiment. If subjects were, indeed, relying on such a combination, then
524
we would expect their reaching endpoints to satisfy two conditions. First, subjects should have
525
touched on average a location between the original, egocentric target presentation location and
526
the shifted, allocentric location. Such an effect can be seen for one example subject in Figure 4.
527
The entire set of raw, target-relative reaching endpoints is divided into four panels according to
Landmark stability important for ego/allocentric weighting
23
528
the quadrant direction of the landmark shift. For example, the upper left panel shows reaching
529
endpoints (filled black circles) that followed landmark shifts to the upper-left quadrant of
530
directions. Not surprisingly, the reaching endpoints here appear quite noisy, first because the
531
scale is quite focused, and more fundamentally these endpoints are influenced by baseline noise,
532
eye movement-induced noise (described above), and motor noise. However, the mean reaching
533
endpoint (thick red circles) for this subject and for a given set of shift directions (i.e. upwards
534
and to the left, upwards and to the right, etc.) was always shifted away from the original target
535
location (origin) toward the line of allocentrically-defined locations (blue arcs). Hence, the
536
landmark shift had a systematic effect in this subject.
537
In order to verify that subjects’ reaching endpoints were, on average, between the
538
egocentric and allocentric locations we first computed the mean allocentric weight for each
539
subject in each experimental condition. Recall, the allocentric weight measure for a given
540
reaching endpoint should be zero if a subject is using only egocentric information, or one if that
541
subject is using only allocentric information. We found overall between-subjects allocentric
542
weight means (±s.e.m.) of Msv_sgs = 0.52±0.09, Msv_lgs = 0.44±0.09, Mlv_sgs = 0.26±0.09, and
543
Mlv_lgs = 0.39±0.08. For each experimental condition we compared the set of 10 subject means to
544
zero, a purely egocentric response, and to one, a purely allocentric response. This set of eight
545
comparisons was performed with standard t-tests, using the stepwise Holm-Bonferroni procedure
546
to correct for multiple comparisons. All tests were found to be significant at the alpha = 0.05
547
level (comparison with zero: psv_sgs = 0.001, psv_lgs = 0.002, plv_sgs = 0.016, plv_lgs = 0.002;
548
comparison with one: psv_sgs = 0.002, psv_lgs = 0.0006, plv_sgs = 0.0001, plv_lgs = 0.0003). Thus, in
549
all conditions subjects touched a point between the original target location and the allocentric
550
location, as expected if both cues were being used.
Landmark stability important for ego/allocentric weighting
551
24
If subjects used visual landmarks as an allocentric cue, then their reaching endpoints
552
should have also covaried with the location of the target relative to the landmarks. For any given
553
trial in our task the location of the visual target relative to the centre of the landmark array (on its
554
first presentation) should predict where the subject touches relative to the landmark array centre
555
on its second presentation. In order to test this we regressed reaching endpoint relative to shifted
556
landmark array centre against target location relative to the original landmark array centre for
557
each subject in each condition. Horizontal and vertical components were regressed separately.
558
This yielded a set of 20 correlation coefficients (two for each subject, one horizontal and one
559
vertical) for each experimental condition. Between-subject means for correlation coefficients
560
within each experimental condition were found to be significantly greater than zero (Holm-
561
Bonferroni corrected p-values ranging from 0.001 to 0.013 after Fisher r-to-z transform),
562
indicating that subjects were indeed using the visual landmarks as allocentric cues (all
563
correlation coefficient values are presented in Supplementary Table S1).
564
The above results confirm that our subjects used both egocentric and allocentric
565
information to different degrees, but they do not tell us how they weighted these factors to
566
choose a particular reaching direction. To test this we had to examine how egocentric-allocentric
567
weighting was affected by our stability and reliability manipulations (see methods). However,
568
first we had to determine the exact effect these manipulations had on variable reaching errors so
569
that we could parameterize our MLE model. Therefore, in the next three sections we present
570
results from our two control experiments before returning to the cue-conflict experiment.
571
572
573
574
Effect of gaze amplitude in the egocentric control
575
gaze trajectory length influences the amount of variability in memory-guided reaching endpoints
One purpose of the egocentric-variability experiment was to test our assumption that the overall
Landmark stability important for ego/allocentric weighting
25
576
when only egocentric information is available. Any increase in variability, we assume, must be
577
indicative of decreased reliability in maintained egocentric information. Raw, target-relative
578
reaching endpoints are shown in Figure 5 for one typical subject. For this subject, reaching
579
endpoints were more variable after large than after small gaze shifts. In fact, the overall reaching
580
variance (defined above) was greater in the large versus small gaze-shift conditions for nine out
581
of ten subjects, with between-subjects means of Msgs = 3.1±0.4cm2 and Mlgs = 5.3±1.3cm2. A
582
priori we would not expect overall reaching variance as defined here to be a normally distributed
583
quantity. Therefore, we compared small and large gaze-shift conditions using a paired-samples
584
Wilcoxon signed-rank test, yielding a significant difference across participants (p = 0.012). Thus,
585
the gaze-shift manipulation appears to have had the expected effect on egocentric information
586
about target location. We will return to this dataset when we use it to predict weighting in our
587
main cue-conflict experiment.
588
589
590
591
Effect of gaze amplitude in the allocentric control
592
varying gaze-shift amplitudes had no effect on allocentric information about target location. In
593
this experiment subjects could generate accurate reaching endpoints only by using allocentric
594
information. To confirm that subjects actually were attempting to reach accurately to the correct
595
landmark-relative target location, as opposed to simply using some other heuristic (e.g. reaching
596
to where they last saw the centre of the landmark array), we performed the same regression
597
procedure as we did in order to verify the use of allocentric information in the main cue-conflict
598
experiment. If subjects were using the visual landmarks as an allocentric cue, then the regression
599
slopes should have been equal to one (because, up to random noise and systematic offsets, the
600
reaching endpoint on a given trial should have been equal to the original landmark-relative
One purpose of the allocentric-variability control experiment was to test our assumption that
Landmark stability important for ego/allocentric weighting
26
601
location of the visual target). Between-subject means for correlation coefficients within each
602
experimental condition were found to be significantly greater than zero for horizontal and
603
vertical directions (Holm-Bonferroni corrected p-values ranging from 0.0002 to 0.006 after
604
Fisher r-to-z transform), while regression slopes were not found to differ significantly from one
605
(p= 1 for all comparisons, except p= 0.3 for the horizontal sv_sgs slope), indicating that subjects
606
were using the visual landmarks as an allocentric cue (all correlation coefficient and slope values
607
are presented in Supplementary Table S1).
608
Sample raw, target-relative reaching data for one subject in the allocentric-variability
609
control experiment is shown in Figure 6. Reaching endpoint variability for this subject appears to
610
be similar in all conditions. In order to quantitatively examine the effect of gaze-shift amplitude
611
on allocentric information across all subjects we calculated the overall reaching variance for each
612
subject in each condition, giving between-subjects means of Msv_sgs = 3.8±0.7cm2, Msv_lgs =
613
4.1±0.8cm2, Mlv_sgs = 3.5±0.7cm2, and Mlv_lgs = 3.7±0.6cm2. Comparing the small and large gaze-
614
shift means within each vibration amplitude condition revealed no significant difference within
615
the small vibration condition (Wilcoxon signed-rank, p = 0.13) or within the large vibration
616
amplitude condition (Wilcoxon signed-rank, p = 0.49). Thus, varying gaze-shift amplitude did
617
not appear to affect reliability of allocentric information about target location. Again, we will
618
return to this data set when we use it to predict reaching endpoints in our cue-conflict experiment.
619
620
621
622
Effect of varying landmark vibration amplitude
623
intention, subjects might judge an unstable landmark to be less useful than a more stable
624
landmark and place less weight on the former compared to the latter when combining this
Varying the vibration amplitude of the landmarks could have two effects. First, as per our
Landmark stability important for ego/allocentric weighting
27
625
information with egocentric information. Second, it could induce noise directly into the
626
allocentric information for reaching, thereby decreasing allocentric reliability (as gaze did for
627
egocentric information). To test this second possibility we compared the overall reaching
628
variance within each gaze-shift amplitude condition of the allocentric-variability control
629
experiment. No significant difference was found within either the small gaze-shift condition
630
(Wilcoxon signed-rank, p = 0.49) or the large gaze-shift condition (Wilcoxon signed-rank, p = 1).
631
Thus, we were successful in varying landmark stability without influencing actual reliability in
632
the allocentric channel. We will return to this dataset when we use it to predict weighting in our
633
main cue-conflict experiment.
634
635
636
637
Weighting as a function of landmark stability and reliability
638
landmarks. Thus, even though landmark vibration amplitude appeared to have no effect on
639
allocentric reliability, we still expected subjects to produce smaller allocentric weights for
640
landmarks with high vibration amplitude (unstable) compared with the low amplitude vibration
641
condition (stable). Given that larger gaze-shift amplitudes produced more egocentric reaching
642
variability, but had no effect on allocentric reliability, we hypothesized that subjects would have
643
generated relatively larger allocentric weights for larger gaze-shifts than for small ones. In order
644
to test these predictions we performed a mixed-model ANOVA on the full set of allocentric
645
weights, with gaze-shift amplitude and landmark vibration amplitude as two, two-level fixed
646
factors, and subject ID as a random factor.
We hypothesized that subjects would place less weight on unstable landmarks relative to stable
647
In Figure 3B, the set of allocentric weights from all trials was divided into four bins for
648
each subject according to the corresponding landmark shift direction (up and to the left, up and
649
to the right, down and to the right, down and to the left) and the means for each direction bin
Landmark stability important for ego/allocentric weighting
28
650
were averaged over subjects. Individual subjects showed quite variable allocentric weightings,
651
with inter-subject variance confirmed to be a significant factor in the data (F(9,4.57) = 7.21, p =
652
0.027), and with allocentric weight means for individual subjects ranging from 0.21 to 0.90 (see
653
Supplementary Figure S2 for a direction-dependent breakdown of individual subject allocentric
654
weights).
655
In Figure 7, the mean allocentric weights are plotted as a function of landmark shift
656
direction, with data separated according to vibration amplitude in the upper circle and data
657
separated according to gaze shift amplitude in the lower circle. From the upper panel it appears
658
as though landmarks with a small vibration amplitude (red curve) had a larger effect on reaching
659
endpoints than did landmarks with a large vibration amplitude (blue curve), especially along a
660
tilted vertical axis. This main effect of landmark vibration amplitude was found to be significant
661
(F(1,9.83) = 6.2, p = 0.032), with the mean between-subjects allocentric weight for large
662
vibration amplitude trials being 0.48 as compared to a mean of 0.33 for the small amplitude
663
vibration trials, but no main effect of gaze-shift amplitude. Thus, visual vibration had the only
664
clear effect on the weighting of allocentric information.
665
Returning to our original model, we used only raw reaching endpoint variability from the
666
main and control experiments in an optimization procedure to derive estimates of egocentric and
667
allocentric reliability, of motor noise, and of the stability parameter, pν ,s (see Appendix). These
668
estimates allowed us to substitute the weight matrices in Equation 2 with MLE estimates from
669
Equation 3 to produce the set of MLE allocentric weights. The results of this procedure are
670
shown in Figure 8. Our model clearly predicted that allocentric information derived from
671
unstable landmarks would be weighed less by subjects in egocentric-allocentric combination
672
than would those derived from stable landmarks, even though stable and unstable landmarks
Landmark stability important for ego/allocentric weighting
673
resulted in similar allocentric reliabilities. This pattern is consistent with the measured data
674
described above.
675
To analyze our data in a more quantitative fashion, we fit the model embodied in
676
Equation 2 to the raw reaching endpoints measured in the main cue-conflict experiment.
677
Calculating direct-fit allocentric weights from this model (as described above) also produced
678
trends that were qualitatively and quantitatively similar to the MLE allocentric weights (see
679
Figure 8). A simple Bootstrap (see Appendix) procedure revealed no statistical difference
680
between the direct-fit allocentric weights and the corresponding MLE allocentric weight
681
predictions. We also tested if our model was able to reproduce specific allocentric weights
682
corresponding to different landmark shift directions (e.g. van Beers et al. 1999). To do this, we
683
calculated direct-fit and MLE allocentric weights as above, but averaged them separately for
684
each full quadrant of shift directions. The model results agreed well with the quadrant-specific
685
effects observed in the experimental data (Supplementary Figure S3).
686
29
In order to verify the importance of the stability parameter, we recalculated the MLE
687
allocentric weights under the constraint that pν ,s = 1 for all conditions. The resulting allocentric
688
weight predictions are also depicted in Figure 8, with the Bootstrapping procedure revealing
689
significant differences between the predictions of this reduced model and the actual data. Thus,
690
allowing a value of pν ,s greater than one (i.e. allowing for a reduced reliance on unstable
691
landmarks regardless of actual reliability) was essential to accurately predicting the empirical
692
data.
693
694
695
696
DISCUSSION
Performance in egocentric and allocentric controls
Landmark stability important for ego/allocentric weighting
30
697
Our control results confirm that subjects were able to reach to remembered targets with
698
reasonable accuracy based on either egocentric or allocentric cues in isolation. This is
699
demonstrated by the fact that between-subjects means for our measure of variable error were at
700
most a little over five square centimeters for any of our control experiments -- small compared
701
with the area over which target location varied. Furthermore, in the allocentric-variability control
702
experiment reaching endpoints within the final landmark array were strongly correlated with
703
target location within the original array, supporting our assumption that subjects would use the
704
landmarks in the intended way.
705
In comparison with other experiments that involve open-loop reaches to remembered
706
targets, endpoint variability in our task was substantially larger than that measured in some
707
experiments (e.g. Krigolson and Heath 2004), but smaller than that measured in others (e.g.
708
Lemay et al. 2004). In addition, Lemay et al. have found that reaches based solely on allocentric
709
information tend to be less variable than reaches based solely on egocentric information. We
710
found no such difference in our results, but our paradigm was also different in numerous aspects.
711
The relative similarity in endpoint variability we found between egocentric and allocentric tasks
712
is consistent with the roughly equal weighting we found between egocentric and allocentric cues
713
in the cue-conflict experiment (see next section).
714
Perhaps more importantly, within our egocentric-variability control experiment we found
715
that larger gaze-shifts during the memory delay induced more variability in reaching endpoints,
716
confirming one assumption behind our experimental design (Prime et al. 2007). This finding is
717
consistent with the idea that egocentric representations of target location are continuously
718
updated each time the eyes move (Henriques et al. 1998; Khan et al. 2005a; Khan et al. 2005b;
719
Medendorp and Crawford 2002; Medendorp et al. 2003b; Merriam et al. 2003). In contrast, in
Landmark stability important for ego/allocentric weighting
31
720
the allocentric-variability control results we found that gaze-shift amplitude had no effect on
721
reach variability, consistent with the general assumption that landmark-relative representations
722
are likely useful because they do not vary with the orientation or configuration of the self in
723
space (e.g. Burgess 2006). The fact that landmark vibration amplitude had little effect on
724
reaching variability in the allocentric task was unexpected, but fortuitous - it meant we were
725
primarily manipulating landmark stability without affecting the reliability of allocentric
726
information and could attribute any change in weighting to the stability heuristic. Each of these
727
factors was then accounted for in our MLE analysis of the cue-conflict experiment, central to the
728
main goals of the experiment.
729
730
Weighting of egocentric and allocentric factors in the cue-conflict experiment
731
As predicted by our MLE model, the shifting visual landmarks in our experiment tended to draw
732
subjects’ reaching responses away from the original target location and towards the allocentric
733
location. Although the weighting factor varied considerably across subjects, both the model
734
predictions and empirical data indicated an overall average weighting of approximately 60%
735
egocentric and 40% allocentric. In some respects it is surprising that allocentric cues had this
736
much effect on reaching, because in debriefing sessions after the cue-conflict experiment
737
subjects indicated that they sometimes detected the allocentric shifts. Indeed, Kording et al.
738
(2007) show that judgments about a target’s perceived location can be heavily influenced by
739
whether a visual cue and an auditory cue to that target’s actual location are perceived as coming
740
from a common source or two separate sources. Given that our MLE model well reproduces our
741
data, this suggests that the weighting of allocentric cues is either hardwired into the visuomotor
742
system or independent of conscious awareness.
Landmark stability important for ego/allocentric weighting
743
32
We cannot rule out the possibility that allocentric weighting in our experiment would
744
have been even higher if the shifts were not at all detectible, but this was not possible to test in
745
our design because it would require cue-shifts to be too small to produce statistical effects
746
against the background noise in our subjects’ performance. We also cannot rule out the
747
possibility that our instructions (ignore the blue dots) did not lessen the effect of the shifting
748
landmarks. Furthermore, Dassonville and Bala (2004) have shown that pointing to egocentric
749
targets can be influenced by an “off-centre” frame that shifts the subject’s estimate of straight
750
ahead. Our frame was much smaller than that of Dassonville & Bala, and other interpretations of
751
their results exist: e.g. de Grave et al. (2002). But in principle, this effect could have produced or
752
influenced the overall shift results we found. However, given the good agreement between our
753
MLE model and our data in all four experimental conditions, and even across shift directions
754
(Supplementary Figure S3), we believe that we have provided strong support for our hypothesis:
755
that humans can and do combine egocentric and allocentric cues to reach toward remembered
756
targets. This finding underscores the brain’s ability to draw upon multiple available information
757
sources when generating behavior, as opposed to simply following some fixed strategy in which
758
only a subset of relevant stimulus information is used in any given context.
759
Our model predicted a main effect of landmark stability on mean reaching endpoints that
760
was in quantitative agreement with the empirically observed value. Thus, egocentric and
761
allocentric visual information appear to be combined by the brain in a stimulus-dependent
762
fashion when generating reaching responses to remembered targets. The fact that a reduced
763
version of our model, one with no stability parameter, could not account for this finding confirms
764
our second hypothesis: that human subjects use heuristic information beyond actual reliability
765
when combining egocentric and allocentric information.
Landmark stability important for ego/allocentric weighting
766
33
Here, our work extends the results of Burgess et al.(2004). These authors had subjects
767
pick which object out of an array of previously viewed objects had been covertly shifted during a
768
brief delay period in which the subject was blindfolded. During this delay, a variety of
769
manipulations could have occurred, including rotation of the circular table on which the objects
770
lay, displacement of an external visual landmark (not on the table), or displacement of the
771
subject via guided walking to a new location around the table. Of relevance here, performance
772
was found to be higher for stationary table/stationary landmark conditions (conditions in which
773
the landmark could be useful) when subjects had not yet been exposed to trials in which the
774
external landmark shifted location between presentation and test (i.e. they seemed to rely more
775
heavily on the landmark when it was thought to be stable). In our work we go further by
776
explicitly examining the cue-combination question and MLE weighting predictions. However,
777
subjects in the Burgess et al. task were likely down-weighting allocentric information because of
778
their past experience with the landmark on previous trial of the experiment. This is not the case
779
in our experiment because the landmarks were equally useful regardless of stability and,
780
therefore, subjects should not have learned to down-weight them. Thus, our subjects must have
781
learned previously or have been hardwired to assume moving landmarks would be less useful.
782
If one only considers performance within the confines of an impoverished laboratory
783
enviroment, this implementation by the brain of a stability heuristic is not optimal behavior. This
784
is because (as we showed) the large amplitude vibration of the cue did not degrade reaching
785
responses. However, the stability parameter in our model was based on the assumption that in
786
natural settings a stability heuristic might actually be optimal. In nature, landmark motion cannot
787
be assumed to result from vibration in one place, but instead is more likely to be motion that
788
would interrupt its validity as a spatial cue, and/or would require extensive temporal averaging to
Landmark stability important for ego/allocentric weighting
34
789
cancel. Thus, our data were consistent with this hypothesis, and from this broader perspective it
790
appears that our subjects’ performance was optimized for behavior in natural, unpredictable
791
settings.
792
Unexpectedly, we did not find any changes in cue weighting when we varied gaze path
793
length (and thus egocentric reliability) during the memory delay. At first glance, this result
794
appeared (even to us) not only counter-intuitive, but to contradict the predictions of our
795
egocentric-variability control experiment in which reaching variability increased by 42%
796
between small and large gaze shift conditions. Based on this, we expected to see a marked
797
difference in cue weighting in these two versions of our cue-conflict experiment. However, this
798
intuition proved false when the data were quantitatively tested against a full MLE model. In brief,
799
the reason is that our initial intuitions were based on a one-dimensional approximation to an
800
inherently two-dimensional quantity. As indicated by the results from our full MLE model, the
801
predicted difference for large versus small gaze-shifts (bottom panel of Figure 8) was simply too
802
small for us to detect in this data set. In general, these results highlight the difficulty of making
803
intuitive predictions when several different interacting factors are at play: quantitative models are
804
required. However, the good agreement between our full MLE model and the data confirms our
805
third hypothesis: that egocentric and allocentric information appear to be combined in a
806
reliability-dependent fashion.
807
808
Comparison to Previous Cue-Combination Studies
809
As discussed in the introduction, numerous studies have investigated the factors that influence
810
egocentric-allocentric combination for reaching. However, few if any have actually examined the
811
role that intrinsic stimulus properties play in the underlying combination rule. By showing that a
Landmark stability important for ego/allocentric weighting
35
812
reliability-dependent MLE model could account quantitatively for our results we have provided
813
further support for the idea that the brain generally combines information in a statistically
814
optimal fashion. However, we have also shown that additional stimulus properties which do not
815
necessarily influence cue reliability must also be taken into account in order to understand fully
816
the cue-combination process that allows for a motor response. We emphasize the motor nature of
817
our task because Knill (2005) has shown that the details of cue-combination do indeed vary
818
based upon whether a response is motor or perceptual. Thus, our findings might not generalize to
819
the latter domain.
820
Of course, it is possible that additional variables that we did not explicitly consider may
821
have contributed to endpoint variability and egocentric-allocentric weighting in our task. For
822
example, movement times and other kinematic variables are often found to correlate in some
823
way with the final reaching endpoints in tasks similar to ours (Heath et al. 2008). Including some
824
of these variables in our model may have further improved the resulting fits, but since our fits are
825
already quite good, we assume that these extra variables do not contain much additional
826
information in our case.
827
Although we found good agreement between our experimental results and our
828
stability/reliability-dependent MLE model, a full Bayesian model would allow for the influence
829
of multiple prior probabilities on various stimulus-related and internally-generated quantities. It
830
has been found in both perceptual (Knill 2007a) and motor (Kording and Wolpert 2004) tasks
831
that the brain does often operate on such Bayesian principles. Thus, it might be interesting to see
832
if subjects could be trained to rely more heavily on the unstable landmarks in our experiment.
833
This might be accomplished by providing trial-to-trial feedback on reaching performance such
834
that subjects were led to believe that their responses were more accurate in the presence of the
Landmark stability important for ego/allocentric weighting
835
unstable landmarks. Finding such a reversal in behaviour would constitute an excellent
836
demonstration that the brain does rely on Bayesian principles when combining egocentric and
837
allocentric information about reaching target location.
36
838
Aside from the stability/reliability-dependent effects seen in our experiment, the fact that
839
subjects could not ignore the visual landmarks even though they were instructed explicitly to do
840
so —a seemingly simple task given that most subjects claimed in a debriefing session after the
841
experiment to have subjectively detected the shift on at least some of the trials— is interesting in
842
itself. Such a finding is consistent with an action-perception dissociation (Goodale and Milner
843
1992). Moreover, this inability to ignore allocentric information could have numerous practical
844
and experimental implications (e.g., in a room that is not completely dark, even barely visible
845
visual geometric information might still be used by the brain and influence results).
846
Another seemingly innocuous stimulus is a fixation point used for gaze position. If not
847
extinguished at the right time, this allocentric cue could influence the behavioral response. For
848
example, reach tends to be biased toward the nearest irrelevant landmark (Diedrichsen et al.
849
2004). This could affect numerous studies, so we will just highlight one relevant example. When
850
humans point or reach toward objects that are not aligned with gaze, the hand tends to overshoot
851
relative to gaze (Bock 1986). This is thought to arise from some unknown error in the visual-
852
motor transformation (Beurze et al. 2006; Henriques et al. 1998). McGuire and Sabes (2009)
853
modeled this by incorporating a mis-estimate of gaze direction relative to the desired reach
854
direction, as well as several other features. The model was successful at independently
855
reproducing most of their experimental data, but it overestimated the effect for larger retinal
856
eccentricities (their Figure 5). However, their continuously illuminated fixation light may have
857
had a distance-dependent influence on performance (Diedrichsen et al. 2004) that was not
Landmark stability important for ego/allocentric weighting
858
accounted for in the model. Removal of the fixation point at the time of reaching, might have
859
improved the fits between their model and their data.
37
860
861
Possible Physiological Mechanisms
862
The neural mechanisms underlying reaching based upon egocentric and allocentric cues remain
863
elusive. Since Goodale and Milner introduced their influential action-perception model posterior
864
parietal regions in the so-called dorsal visual stream have become strongly associated with
865
visually-guided action, while temporal regions in the ventral stream have become associated with
866
visual perception. However, delayed action based on remembered targets has also been argued to
867
depend on the ventral stream (Goodale et al. 2004). Moreover, various emerging lines of
868
evidence suggest that the dorsal stream processes egocentric visuospatial information, while the
869
ventral stream deals more with allocentric information - whether for action or perception (Carey
870
et al. 2006; Schenk 2006). In line with this idea, Thaler and Todd (2009) have shown that the
871
specific reference frame (egocentric or allocentric) used for a task affects response variance, but
872
that such variance is unaffected by whether the task is related to action or perception. Other
873
experiments suggest that egocentric and allocentric signals appear in both streams. For example,
874
neurons in the lateral intraparietal area of the monkey show rudimentary feature responses
875
(Sereno and Maunsell 1998). However, it appears that highly detailed object-relative spatial
876
information is represented in ventrolateral temporo-occipital areas comprising the ventral stream
877
(Brincat and Connor 2004; Pasupathy and Connor 2002).
878
Assuming that the initial detailed analysis of allocentric information is performed in the
879
ventral visual stream, it still must enter the ‘dorsal stream’ parieto-frontal loop at some point to
880
influence motor behavior. Consistent with this, egocentric and allocentric judgment tasks have
Landmark stability important for ego/allocentric weighting
38
881
been shown to produce elevated levels of activity in the right human posterior parietal cortex
882
(Galati et al. 2000; Zaehle et al. 2007). In addition, monkeys trained to perform visuospatial
883
tasks involving both egocentic and allocentric elements showed clear object-centered neural
884
responses in pre-frontal cortex (Olson and Tremblay 2000) and in posterior parietal cortex
885
(Chafee et al. 2007; Crowe et al. 2008; Sabes et al. 2002). Interestingly, Crowe et al.’s results
886
suggest that egocentric representations of target location are formed in parietal cortex
887
(specifically area 7a) before object-based ones. This could imply that the egocentric
888
representations are being transformed into allocentric ones in parietal cortex or that the
889
allocentric information is arriving there from elsewhere, possibly from ventral stream regions, as
890
we describe below.
891
The possibility that object-based allocentric information flows from the ventral visual
892
stream to posterior parietal cortex is similar to the principle underlying a neural network model
893
proposed by Byrne et al. (2007). Within this model, landmark-based allocentric representations
894
of navigable space are initially found in medial temporal areas but must be transformed into
895
egocentric representations via posterior parietal cortex in order to be used for path planning,
896
mental imagery, etc (for a review of evidence supporting this principle, see Vann et al. (2009)).
897
Indeed, Committeri et al. (2004) have shown that egocentric, landmark-based allocentric, and
898
object-based allocentric tasks all produced activation in parieto-frontal areas, while landmark-
899
based allocentric tasks produced activation in ventromedial temporal areas, and object-based
900
allocentric tasks produced activation in ventrolateral areas of temporal and occipital cortices.
901
Hence, we speculate that object-based allocentric representations in our task were initially
902
formed in the ventral visual stream and then transferred to the parieto-frontal loop for visuomotor
903
control via posterior parietal regions. This transfer could occur directly, or via reciprocal
Landmark stability important for ego/allocentric weighting
904
recurrent connections between the dorsal stream and the ventral stream at the level of occipital
905
cortex (Merriam et al. 2007; Prime et al. 2008).
906
39
Once allocentric information enters the parieto-frontal loop, it might either be combined
907
immediately with egocentric information to generate a single representation of target location
908
that is maintained over memory delays, or it might be maintained there separately until a
909
reaching response is required. Whichever the case, numerous studies indicate that a dorsolateral
910
prefrontal cortex-posterior parietal cortex loop is essential in the maintenance of spatial memory
911
in a wide variety of working memory tasks (e.g. Chafee and Goldman-Rakic 1998; Koch et al.
912
2005). In the case of saccade targets both egocentric (Dassonville et al. 1992; Schlag and Schlag-
913
Rey 1987; Thier and Andersen 1998, 1996) and allocentric (Olson and Gettner 1995; Sabes et al.
914
2002) representations of target location have been found in the parieto-frontal loop. In the case of
915
reaching targets a region of human parietal cortex, tentatively referred to as human PRR, has
916
been shown to support gaze-centered (i.e. egocentric) representations of reach target location
917
during memory intervals (Medendorp et al. 2003a). However, to our knowledge allocentric
918
representations of reach target location have not been found in parieto-frontal circuitry (but see
919
Snyder et al. 1998), so it is difficult to comment with any more certainty.
920
Another question is the physiological mechanism for the stability heuristic used in our
921
model. It is likely in our experiment that cue vibration was detected by the MT/MST complex,
922
which is exquisitely sensitive to visual motion and projects to both parietal and frontal movement
923
areas (eg. Ilg 2008). But again, it is highly uncertain where this information enters the parieto-
924
frontal loop. One speculative possibility is that egocentric and allocentric representations of
925
target location are integrated in premotor cortex. This is suggested by the work of Verhagen et al.
Landmark stability important for ego/allocentric weighting
40
926
(2008) who show that the ventral premotor region seems to be involved in integrating perceptual
927
information from the ventral stream into the grasp plan.
928
929
Conclusions
930
In summary, we have provided the first demonstration using a cue-conflict paradigm that
931
egocentric and allocentric visual information are combined in a stimulus-dependent fashion for
932
generating reaching movements to visual targets. Perhaps most importantly, we have shown that
933
the underlying combination rule seems to depend on heuristics beyond an accounting for actual
934
cue reliability. This finding is important because it shows that although the brain can make
935
intelligent use of the various sources of information that are available to it, it might also depend
936
to an extent on certain inflexible “rules of thumb”. We have also shown that the underlying
937
combination process, whatever its exact nature, is obligatory and cannot easily be overridden by
938
conscious processes based on perception.
939
940
941
942
943
944
945
In order to calculate estimates for covariance matrices representing egocentric and allocentric
946
reliability and motor noise, and an estimate of the stability parameter, pν ,s , we first assumed that
947
subjects in the two variability control experiments based their reaching responses on similar
948
estimates as they did in the main experiment. Thus, a reaching endpoint in the allocentric-
949
variability experiment would be given by
950
rpa = Mνa , s rˆa;aν , s + bνa , s + ε m
s
APPENDIX
,
(4)
Landmark stability important for ego/allocentric weighting
41
951
where symbols have similar meanings as in Equations 1 & 2, and ν has an identical meaning
952
because the same experimental conditions were used in the allocentric-variability control as in
953
the main experiment. The superscript ‘a’ is used on some of the variables in Equation 4 to
954
indicate that their values are not necessarily the same as the equivalent variables in Equations 1
955
& 2. Also, since subjects were exposed to identical reliability manipulations and stimulus
956
characteristics in the allocentric-variability control experiment as in the main experiment, we
957
assumed that rˆa;aν ,s had the same covariance as allocentric information in the main experiment,
958
namely Σa;ν ,s . For the egocentric-variability control experiment, we have
959
rpe = Mνe , s rˆe;e ν , s + bνe , s + ε m
s
960
where ν refers only to small versus large gaze shifts since there were no landmarks in this
961
control experiment. However, we did assume that rˆe;e ν , s was distributed with the same covariance
962
as egocentric information in the main experiment. That is, we assumed that Σe;sv _ sgs, s and
963
Σ e;lv _ sgs , s
964
similarly for the sv_lgs and lv_lgs conditions. Hence, we also refer to the covariance of rˆe;e ν ,s as
965
Σ e;ν ,s .
966
967
,
(5)
from the main experiment were equal to Σe;sgs,s from the control experiment, and
Rewriting Equations 4 & 5 in the same way that we converted Equation 1 into Equation 2
968
gives
969
rpa = Mνa , s raa + bνa , s + ενa, s ,
970
and
971
rpe = Mνe , s ree + bνe , s + ενe, s .
(6)
(7)
Landmark stability important for ego/allocentric weighting
42
972
Fitting Equations 2, 6, and 7 yielded a set of residuals for each subject and each condition in each
973
experiment. We denote the covariance matrices corresponding to residuals from the main cue-
974
conflict, the allocentric-variability control, and the egocentric-variability control experiments
975
by Cν ,s , Cνa ,s , and Cνe ,s respectively. From the right hand sides of Equations 1, 4, and 5 we
976
calculated the expected values of these covariances, giving
977
Cν , s = Mν , s Wν , s Σ e;ν , s Mν , s Wν , s
978
Cνa , s = Mνa , s Σ a;ν , s Mνa , s
979
and
980
Cνe , s = Mνe , s Σ e;ν , s Mνe , s
981
where the superscript T is matrix transpose and I is the identity matrix.
(
982
(
)
T
(
)
T
)
T
(
)
(
(
+ Mν , s I − Wν , s Σa;ν , s Mν , s I − Wν , s
))
T
+ Σm
s ,
(8)
+ Σm
s ,
(9)
+ Σm
s ,
(10)
Were it not for motor error we could simply use Equations 9 & 10 to solve for Σa;ν ,s and
in the main experiment. However, we require an estimate for Σ ms and for the stability
983
Σ e;ν ,s
984
parameter. We obtained this by presupposing that subjects were using a reliability-dependent
985
MLE combination of egocentric and allocentric information. By doing this we could solve
986
Equations 9 & 10 for Σa;ν ,s and Σe;ν ,s , substitute these values into Equation 3 & 8, and then
987
substitute Equation 3 into Equation 8. This yielded a set of four two-by-two matrix Equations of
988
the form
989
Fν Σ m
s , pν , s = 0 ,
(
)
(11)
Landmark stability important for ego/allocentric weighting
43
990
where Fν is a matrix function depending on experimental condition. Thus, in order to find an
991
estimate of Σ ms , we numerically minimized the objective function,
992
∑∑ ⎡⎢⎣ Fν i, j ( Σms , pν ,s )⎤⎥⎦
2
,
(12)
ν i, j
993
with respect to plv_sgs,s = plv_lgs,s, = ps and the components of Σ ms . Note, as describe above we
994
have taken pν , s = 1 for both small vibration, stable landmark conditions.
995
Once we obtained estimates of Σ ms and pν ,s for each subject, we used Equations 9 and 10
996
to calculate Σa;ν ,s and Σe;ν ,s for each subject in each condition. These values were then used with
997
Equation 3 to generate weight matrices for Equation 2. As described in the Methods sections, we
998
then used Equation 2 to produce the set of MLE allocentric weights. The entire optimization
999
procedure, including the calculation of Σa;ν ,s and Σe;ν ,s was performed under the constraint
1000
that Σ ms , Σa;ν ,s , and Σe;ν ,s had to be real, symmetric and positive definite (i.e. they had to be valid
1001
covariance matrices).
1002
In order to generate confidence intervals for the differences between the MLE allocentric
1003
weights and the direct-fit weights in each experimental condition we first calculated the
1004
difference between each individual subject’s MLE and direct-fit mean in that condition. We then
1005
re-sampled this set of differences (with replacement) 10000 times and to produce 95%
1006
confidence intervals for the mean MLE/direct-fit allocentric weight difference in each condition.
1007
This procedure revealed no significant differences between our model and the data.
1008
In order to investigate the importance of the stability parameter, we set plv_sgs,s = plv_lgs,s,
1009
= 1 in Equation 3 so that only the components of Σ ms varied in Equation 12. Bootstrapping was
Landmark stability important for ego/allocentric weighting
44
1010
performed for this reduced model in an identical fashion to the full model, revealing significant
1011
differences between MLE and direct-fit allocentric weights.
1012
1013
1014
1015
This research was supported by a grant from the Canadian Institutes of Health Research (CIHR).
1016
Pat Byrne was supported by the CIHR Strategic Training Program in Vision Health Research.
1017
Doug Crawford was supported by a CIHR Canada Research Chair.
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
ACKNOWLEDGMENTS
Landmark stability important for ego/allocentric weighting
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
45
REFERENCES
Allen DM. Relationship between Variable Selection and Data Augmentation and a Method for
Prediction. Technometrics 16: 125-127, 1974.
Ariff G, Donchin O, Nanayakkara T, and Shadmehr R. A real-time state predictor in motor
control: study of saccadic eye movements during unseen reaching movements. J Neurosci 22:
7721-7729, 2002.
Batista AP, Buneo CA, Snyder LH, and Andersen RA. Reach plans in eye-centered
coordinates. Science 285: 257-260, 1999.
Battaglia PW, Jacobs RA, and Aslin RN. Bayesian integration of visual and auditory signals
for spatial localization. J Opt Soc Am A Opt Image Sci Vis 20: 1391-1397, 2003.
Beurze SM, Van Pelt S, and Medendorp WP. Behavioral reference frames for planning human
reaching movements. J Neurophysiol 96: 352-362, 2006.
Biegler R and Morris R. Landmark stability: studies exploring whether the perceived stability
of the environment influences spatial representation. J Exp Biol 199: 187-193, 1996a.
Biegler R and Morris RG. Landmark stability is a prerequisite for spatial but not discrimination
learning. Nature 361: 631-633, 1993.
Biegler R and Morris RG. Landmark stability: further studies pointing to a role in spatial
learning. Q J Exp Psychol B 49: 307-345, 1996b.
Blohm G and Crawford JD. Computations for geometrically accurate visually guided reaching
in 3-D space. J Vis 7: 4 1-22, 2007.
Bock O. Contribution of retinal versus extraretinal signals towards visual localization in goaldirected movements. Exp Brain Res 64: 476-482, 1986.
Brainard DH. The Psychophysics Toolbox. Spat Vis 10: 433-436, 1997.
Bridgeman B, Peery S, and Anand S. Interaction of cognitive and sensorimotor maps of visual
space. Percept Psychophys 59: 456-469, 1997.
Brincat SL and Connor CE. Underlying principles of visual shape selectivity in posterior
inferotemporal cortex. Nat Neurosci 7: 880-886, 2004.
Brouwer AM and Knill DC. Humans use visual and remembered information about object
location to plan pointing movements. J Vis 9: 24 21-19, 2009.
Buneo CA, Jarvis MR, Batista AP, and Andersen RA. Direct visuomotor transformations for
reaching. Nature 416: 632-636, 2002.
Burgess N. Spatial memory: how egocentric and allocentric combine. Trends Cogn Sci 10: 551557, 2006.
Burgess N, Spiers HJ, and Paleologou E. Orientational manoeuvres in the dark: dissociating
allocentric and egocentric influences on spatial memory. Cognition 94: 149-166, 2004.
Byrne P, Becker S, and Burgess N. Remembering the past and imagining the future: a neural
model of spatial memory and imagery. Psychol Rev 114: 340-375, 2007.
Carey DP, Dijkerman HC, Murphy KJ, Goodale MA, and Milner AD. Pointing to places
and spaces in a patient with visual form agnosia. Neuropsychologia 44: 1584-1594, 2006.
Carlton LG. Processing visual feedback information for movement control. J Exp Psychol Hum
Percept Perform 7: 1019-1030, 1981.
Carrozzo M, Stratta F, McIntyre J, and Lacquaniti F. Cognitive allocentric representations
of visual space shape pointing errors. Exp Brain Res 147: 426-436, 2002.
Landmark stability important for ego/allocentric weighting
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
46
Chafee MV, Averbeck BB, and Crowe DA. Representing spatial relationships in posterior
parietal cortex: single neurons code object-referenced position. Cereb Cortex 17: 2914-2932,
2007.
Chafee MV and Goldman-Rakic PS. Matching patterns of activity in primate prefrontal area 8a
and parietal area 7ip neurons during a spatial working memory task. J Neurophysiol 79: 29192940, 1998.
Committeri G, Galati G, Paradis AL, Pizzamiglio L, Berthoz A, and LeBihan D. Reference
frames for spatial cognition: different brain areas are involved in viewer-, object-, and landmarkcentered judgments about object location. J Cogn Neurosci 16: 1517-1535, 2004.
Crawford JD, Medendorp WP, and Marotta JJ. Spatial transformations for eye-hand
coordination. J Neurophysiol 92: 10-19, 2004.
Crowe DA, Averbeck BB, and Chafee MV. Neural ensemble decoding reveals a correlate of
viewer- to object-centered spatial transformation in monkey parietal cortex. J Neurosci 28: 52185228, 2008.
Dassonville P and Bala JK. Perception, action, and Roelofs effect: a mere illusion of
dissociation. PLoS Biol 2: e364, 2004.
Dassonville P, Schlag J, and Schlag-Rey M. The frontal eye field provides the goal of saccadic
eye movement. Exp Brain Res 89: 300-310, 1992.
de Grave DD, Brenner E, and Smeets JB. Are the original Roelofs effect and the induced
Roelofs effect caused by the same shift in straight ahead? Vision Res 42: 2279-2285, 2002.
de Grave DD, Brenner E, and Smeets JB. Illusions as a tool to study the coding of pointing
movements. Exp Brain Res 155: 56-62, 2004.
Diedrichsen J, Werner S, Schmidt T, and Trommershauser J. Immediate spatial distortions
of pointing movements induced by visual landmarks. Percept Psychophys 66: 89-103, 2004.
Ernst MO and Banks MS. Humans integrate visual and haptic information in a statistically
optimal fashion. Nature 415: 429-433, 2002.
Flanagan JR, King S, Wolpert DM, and Johansson RS. Sensorimotor prediction and memory
in object manipulation. Can J Exp Psychol 55: 87-95, 2001.
Flanagan JR, Terao Y, and Johansson RS. Gaze behavior when reaching to remembered
targets. J Neurophysiol 100: 1533-1543, 2008.
Flanagan JR, Vetter P, Johansson RS, and Wolpert DM. Prediction precedes control in motor
learning. Curr Biol 13: 146-150, 2003.
Galati G, Lobel E, Vallar G, Berthoz A, Pizzamiglio L, and Le Bihan D. The neural basis of
egocentric and allocentric coding of space in humans: a functional magnetic resonance study.
Exp Brain Res 133: 156-164, 2000.
Glover S and Dixon P. A step and a hop on the Muller-Lyer: illusion effects on lower-limb
movements. Exp Brain Res 154: 504-512, 2004.
Goodale MA and Milner AD. Separate visual pathways for perception and action. Trends
Neurosci 15: 20-25, 1992.
Goodale MA, Westwood DA, and Milner AD. Two distinct modes of control for objectdirected action. Prog Brain Res 144: 131-144, 2004.
Hanisch C, Konczak J, and Dohle C. The effect of the Ebbinghaus illusion on grasping
behaviour of children. Exp Brain Res 137: 237-245, 2001.
Hay L and Redon C. Response delay and spatial representation in pointing movements.
Neurosci Lett 408: 194-198, 2006.
Landmark stability important for ego/allocentric weighting
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
47
Heath M, Maraj A, Godbolt B, and Binsted G. Action without awareness: reaching to an
object you do not remember seeing. PLoS One 3: e3539, 2008.
Henriques DY, Klier EM, Smith MA, Lowy D, and Crawford JD. Gaze-centered remapping
of remembered visual space in an open-loop pointing task. J Neurosci 18: 1583-1594, 1998.
Ilg UJ. The role of areas MT and MST in coding of visual motion underlying the execution of
smooth pursuit. Vision Res 48: 2062-2069, 2008.
Jeffery KJ. Learning of landmark stability and instability by hippocampal place cells.
Neuropharmacology 37: 677-687, 1998.
Keele SW and Posner MI. Processing of visual feedback in rapid movements. Journal of
Experimental Psychology 77: 155-158, 1968.
Khan AZ, Pisella L, Rossetti Y, Vighetto A, and Crawford JD. Impairment of gaze-centered
updating of reach targets in bilateral parietal-occipital damaged patients. Cereb Cortex 15: 15471560, 2005a.
Khan AZ, Pisella L, Vighetto A, Cotton F, Luaute J, Boisson D, Salemme R, Crawford JD,
and Rossetti Y. Optic ataxia errors depend on remapped, not viewed, target location. Nat
Neurosci 8: 418-420, 2005b.
Knill DC. Learning Bayesian priors for depth perception. J Vis 7: 13, 2007a.
Knill DC. Reaching for visual cues to depth: the brain combines depth cues differently for motor
control and perception. J Vis 5: 103-115, 2005.
Knill DC. Robust cue integration: a Bayesian model and evidence from cue-conflict studies with
stereoscopic and figure cues to slant. J Vis 7: 5 1-24, 2007b.
Knill DC and Saunders JA. Do humans optimally integrate stereo and texture information for
judgments of surface slant? Vision Res 43: 2539-2558, 2003.
Koch G, Oliveri M, Torriero S, Carlesimo GA, Turriziani P, and Caltagirone C. rTMS
evidence of different delay and decision processes in a fronto-parietal neuronal network activated
during spatial working memory. Neuroimage 24: 34-39, 2005.
Kording KP, Beierholm U, Ma WJ, Quartz S, Tenenbaum JB, and Shams L. Causal
inference in multisensory perception. PLoS One 2: e943, 2007.
Kording KP and Wolpert DM. Bayesian integration in sensorimotor learning. Nature 427: 244247, 2004.
Krigolson O, Clark N, Heath M, and Binsted G. The proximity of visual landmarks impacts
reaching performance. Spat Vis 20: 317-336, 2007.
Krigolson O and Heath M. Background visual cues and memory-guided reaching. Hum Mov
Sci 23: 861-877, 2004.
Lemay M, Bertram CP, and Stelmach GE. Pointing to an allocentric and egocentric
remembered target. Motor Control 8: 16-32, 2004.
McGuire LM and Sabes PN. Sensory transformations and the use of multiple reference frames
for reach planning. Nat Neurosci 12: 1056-1061, 2009.
Medendorp WP and Crawford JD. Visuospatial updating of reaching targets in near and far
space. Neuroreport 13: 633-636, 2002.
Medendorp WP, Goltz HC, Vilis T, and Crawford JD. Gaze-centered updating of visual
space in human parietal cortex. J Neurosci 23: 6209-6214, 2003a.
Medendorp WP, Tweed DB, and Crawford JD. Motion parallax is computed in the updating
of human spatial memory. J Neurosci 23: 8135-8142, 2003b.
Merriam EP, Genovese CR, and Colby CL. Remapping in human visual cortex. J
Neurophysiol 97: 1738-1755, 2007.
Landmark stability important for ego/allocentric weighting
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
48
Merriam EP, Genovese CR, and Colby CL. Spatial updating in human parietal cortex. Neuron
39: 361-373, 2003.
Neely KA, Tessmer A, Binsted G, and Heath M. Goal-directed reaching: movement strategies
influence the weighting of allocentric and egocentric visual cues. Exp Brain Res 186: 375-384,
2008.
Neggers SF, Scholvinck ML, van der Lubbe RH, and Postma A. Quantifying the interactions
between allo- and egocentric representations of space. Acta Psychol (Amst) 118: 25-45, 2005.
Niemeier M, Crawford JD, and Tweed DB. Optimal transsaccadic integration explains
distorted spatial perception. Nature 422: 76-80, 2003.
Obhi SS and Goodale MA. The effects of landmarks on the performance of delayed and realtime pointing movements. Exp Brain Res 167: 335-344, 2005.
Olson CR and Gettner SN. Object-centered direction selectivity in the macaque supplementary
eye field. Science 269: 985-988, 1995.
Olson CR and Tremblay L. Macaque supplementary eye field neurons encode object-centered
locations relative to both continuous and discontinuous objects. J Neurophysiol 83: 2392-2411,
2000.
Pasupathy A and Connor CE. Population coding of shape in area V4. Nat Neurosci 5: 13321338, 2002.
Paulignan Y, MacKenzie C, Marteniuk R, and Jeannerod M. Selective perturbation of visual
input during prehension movements. 1. The effects of changing object position. Exp Brain Res
83: 502-512, 1991.
Pelli DG. The VideoToolbox software for visual psychophysics: transforming numbers into
movies. Spat Vis 10: 437-442, 1997.
Prime SL, Niemeier M, and Crawford JD. Transsaccadic integration of visual features in a
line intersection task. Exp Brain Res 169: 532-548, 2006.
Prime SL, Tsotsos L, Keith GP, and Crawford JD. Visual memory capacity in transsaccadic
integration. Exp Brain Res 180: 609-628, 2007.
Prime SL, Vesia M, and Crawford JD. Transcranial magnetic stimulation over posterior
parietal cortex disrupts transsaccadic memory of multiple objects. J Neurosci 28: 6938-6949,
2008.
Robinson DA. The use of control systems analysis in the neurophysiology of eye movements.
Annu Rev Neurosci 4: 463-503, 1981.
Sabes PN, Breznen B, and Andersen RA. Parietal representation of object-based saccades. J
Neurophysiol 88: 1815-1829, 2002.
Schenk T. An allocentric rather than perceptual deficit in patient D.F. Nat Neurosci 9: 13691370, 2006.
Schlag J and Schlag-Rey M. Does microstimulation evoke fixed-vector saccades by generating
their vector or by specifying their goal? Exp Brain Res 68: 442-444, 1987.
Sereno AB and Maunsell JH. Shape selectivity in primate lateral intraparietal cortex. Nature
395: 500-503, 1998.
Sheth BR and Shimojo S. Extrinsic cues suppress the encoding of intrinsic cues. J Cogn
Neurosci 16: 339-350, 2004.
Smeets JB, van den Dobbelsteen JJ, de Grave DD, van Beers RJ, and Brenner E. Sensory
integration does not lead to sensory calibration. Proc Natl Acad Sci U S A 103: 18781-18786,
2006.
Landmark stability important for ego/allocentric weighting
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
49
Snyder LH, Grieve KL, Brotchie P, and Andersen RA. Separate body- and world-referenced
representations of visual space in parietal cortex. Nature 394: 887-891, 1998.
Taylor JR. An introduction to error analysis : the study of uncertainties in physical
measurements. Sausalito, Calif.: University Science Books, 1997.
Thaler L and Todd JT. The use of head/eye-centered, hand-centered and allocentric
representations for visually guided hand movements and perceptual judgments.
Neuropsychologia 47: 1227-1244, 2009.
Thier P and Andersen RA. Electrical microstimulation distinguishes distinct saccade-related
areas in the posterior parietal cortex. J Neurophysiol 80: 1713-1735, 1998.
Thier P and Andersen RA. Electrical microstimulation suggests two different forms of
representation of head-centered space in the intraparietal sulcus of rhesus monkeys. Proc Natl
Acad Sci U S A 93: 4962-4967, 1996.
van Beers RJ, Sittig AC, and Gon JJ. Integration of proprioceptive and visual positioninformation: An experimentally supported model. J Neurophysiol 81: 1355-1364, 1999.
van der Kamp J, Savelsbergh G, and Smeets J. Multiple information sources in interceptive
timing. Human Movement Science 16: 787-821, 1997.
Vann SD, Aggleton JP, and Maguire EA. What does the retrosplenial cortex do? Nat Rev
Neurosci 10: 792-802, 2009.
Vaziri S, Diedrichsen J, and Shadmehr R. Why does the brain predict sensory consequences
of oculomotor commands? Optimal integration of the predicted and the actual sensory feedback.
J Neurosci 26: 4188-4197, 2006.
Verhagen L, Dijkerman HC, Grol MJ, and Toni I. Perceptuo-motor interactions during
prehension movements. J Neurosci 28: 4726-4735, 2008.
Zaehle T, Jordan K, Wustenberg T, Baudewig J, Dechent P, and Mast FW. The neural basis
of the egocentric and allocentric spatial frame of reference. Brain Res 1137: 92-103, 2007.
Zelaznik HZ, Hawkins B, and Kisselburgh L. Rapid visual feedback processing in singleaiming movements. J Mot Behav 15: 217-236, 1983.
Landmark stability important for ego/allocentric weighting
50
1277
1278
1279
FIGURE CAPTIONS
Figure 1: A) Cue-conflict experiment. Subjects were presented briefly with a to-be-remembered
1280
target (yellow) surrounded by four vibrating landmarks (blue). At the end of a memory delay
1281
following target and landmark offset the landmarks reappeared at a slightly shifted location.
1282
After the second landmark offset, subjects reached to touch the remembered target location. The
1283
fixation cross made two jumps during the memory delay in order to induce gaze-shifts of small
1284
or large amplitude. B) Egocentric-variability control experiment. Conventions are identical to the
1285
main experiment, but with no visual landmarks. C) Allocentric-variability control experiment.
1286
Conventions are identical to the main experiment, but the landmark shift is much larger and
1287
subjects were to reach based on new landmark-relative target location.
1288
Figure 2: A) Cue-conflict experiment. B) Egocentric-variability control experiment. Same as
1289
cue-conflict task without landmarks. C) Allocentric-variability control experiment. Same as cue-
1290
conflict, but landmark shift was large and subjects were to reach based on new landmark-relative
1291
target location. Variable reaching error in all three experiments was assumed to arise partly from
1292
a common motor source with covariance, Σ m . Within the egocentric and allocentric control
1293
experiments additional variability was assumed to come from representational sources with
1294
covariances given by Σe and Σa , respectively. Within the cue-conflict experiment additional
1295
variability was assumed to come from a combination of egocentric and allocentric
1296
representational sources that depends on the stability parameter, p. D) Assuming our MLE model
1297
is accurate, it provides a way to recover Σe , Σa , Σ m and p from the observed variable error in all
1298
three experiments. Based on the resulting values for Σe , Σa , and p the model is able to predict
1299
egocentric-allocentric weighting in the cue-conflict experiment. Note that the mean weighting of
Landmark stability important for ego/allocentric weighting
51
1300
reaching responses in the various cue-conflict experimental conditions constitutes a data set that
1301
has no a priori relationship with reaching variability.
1302
Figure 3: A) Transformation procedure. After correcting all reaching endpoints for systematic
1303
reaching bias (see Methods) each response (small red circle) was transformed by translating,
1304
rotating, and scaling its position vector so that the original target (solid orange disc) would be at
1305
the origin of the new coordinate system and the allocentric location (large, dashed blue circle)
1306
would be at the (1,0) location. B) Overall effect of landmark according to shift direction. The
1307
orange disc at the center of the circular plot represents original target location, while the blue
1308
outer circle represents the set of possible allocentric locations. The mean allocentric weight for a
1309
given direction is represented by the intersection point of the solid black curve with the dashed
1310
gray line segment corresponding to that direction. The solid curve itself is a cubic spline
1311
interpolation of these intersection points and simply serves as a guide to the eye.
1312
Figure 4: Target-relative reaching endpoints for one subject in the cue-conflict experiment
1313
divided according to the direction of landmark shift. For example, if on a given trial the
1314
landmarks shifted upwards and to the left relative to their initial position, then the target-relative
1315
reaching endpoint for that trial is plotted in the upper left panel as a filled black circle. The
1316
orange disc at the origin of each panel represents the target location on all trials, while the dashed
1317
blue arcs in each panel represent the possible allocentric locations (described in the text). The
1318
large empty red circle in each panel represents the means of the target-relative reaching data for
1319
that panel/shift direction.
1320
Figure 5: Top two panels show sample small gaze shift and large gaze shift eye movement traces
1321
for one subject in the egocentric-variability control experiment. Subject gaze always starts at
1322
centre. Bottom left/right panel: All target-relative reaching endpoints generated by this subject in
Landmark stability important for ego/allocentric weighting
52
1323
the small/large gaze shift condition. Each filled circle represents one target-relative reaching
1324
endpoint, while the orange disc at the center of the panel represents the target location for all
1325
trials.
1326
Figure 6: All target-relative reaching endpoints generated by this subject in all four conditions of
1327
the allocentric-variability control experiment. Each filled circle represents one target-relative
1328
reaching endpoint, while the orange disc at the center of the panel represents the target location
1329
for all trials. This subject shows a slight leftward reaching bias, but no such effect is seen across
1330
subjects.
1331
Figur 7: Effect of experimental manipulations on allocentric weights. The orange disc at the
1332
center of each subplot represents target location, while the dashed blue circle represents the set of
1333
allocentric locations. Each dashed line segment corresponds to an angular bin, and its
1334
intersection with a solid, closed contour represents the mean allocentric weight for landmark
1335
shift directions in that bin averaged over subjects.
1336
Figur 8: Top panel: Solid, blue bars are between-subjects means for direct-fit allocentric weights
1337
in the small and large landmark vibration conditions. Error bars are between-subjects S.E.M.
1338
Solid, red bars are predictions from our full reliability-dependent MLE model. Hollow bars are
1339
corresponding predictions from the reduced model, without the stability parameter. Bottom panel:
1340
Same as top, but with data grouped according to gaze shift amplitude. Statistical differences are
1341
at the α = 0.05 level and are derived from Bootstrapping (see Appendix).
1342
1343
1344
1345
Landmark stability important for ego/allocentric weighting
53
1346
1347
1348
Supplementary Material
1349
conflict experiment. Panels a, c, e and g each show vertical finger position and gaze direction
1350
from the end of the initial fixation period to the end of the reaching interval for one sv_lgs,
1351
lv_sgs, sv_sgs, and lv_lgs trial, respectively. The thinner curve starting at 0 deg represents gaze
1352
direction, while the thicker curve starting at 35 deg down represents the finger position as
1353
projected onto the plane containing the display screen. The empty rectangles represent the
1354
vertical locations and time periods during which the fixation cross was present (the initial
1355
fixation cross at 0 deg is omitted for clarity). The gray-filled solid rectangles represent the
1356
vertical location of the egocentric target while it was visible, while the gray-filled dashed
1357
rectangles represent the location that the target would have had if it had reappeared with the
1358
shifted landmarks (allocentric location). Panels b, d, f, and h show the same trials, but from the
1359
subjects perspective. Again, the thinner curve starting at 0 deg represents gaze direction, while
1360
the thicker curve entering from the lower right represents the finger position. Here, the filled
1361
circle represents the original target location, while the empty circle represents the allocentric
1362
location.
1363
Supplementary Figure S2: Overall effect of landmark shift by subject. This Figure is identical to
1364
Figure 3B except that allocentric weight data is not averaged over subjects, but rather plotted
1365
separately for each subject (one circular plot per subject).
1366
Supplementary Figure S3: Effect of experimental manipulations on direct-fit and MLE-predicted
1367
allocentric weights grouped by landmark shift direction. The orange disc at the center of each
1368
subplot represents target location, while the dashed blue circle represents the set of allocentric
1369
locations. Each dashed line segment corresponds to an angular bin of 90 degree width, and its
Supplementary Figure S1: Sample finger and eye movement traces for one subject in the cue-
Landmark stability important for ego/allocentric weighting
1370
intersection with a solid, closed contour represents the mean allocentric weight for landmark
1371
shift directions in that bin averaged over subjects.
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
54
Cue reliability and landmark stability determine allocentric weighting
1
Supplementary Table S1. Relationship between second presentation landmark-relative reaching
endpoints and first presentation landmark-relative target location.
Condition
Mean Correlation/Slope
Holm-Bonferroni p-value
Cue-Conflict Experiment Correlation Coefficients
Horizontal
sv_sgs
sv_lgs
lv_sgs
lv_lgs
0.36±0.06
0.35±0.05
0.23±0.07
0.35±0.06
0.003
0.001
0.013
0.004
sv_sgs
sv_lgs
lv_sgs
lv_lgs
0.29±0.07
0.26±0.07
0.33±0.07
0.28±0.06
0.007
0.008
0.005
0.005
Vertical
Allocentric-Variability Control Experiment Correlation Coefficients
Horizontal
sv_sgs
sv_lgs
lv_sgs
lv_lgs
0.36±0.07
0.40±0.07
0.39±0.09
0.45±0.05
0.006
0.006
0.006
0.0002
sv_sgs
sv_lgs
lv_sgs
lv_lgs
0.43±0.05
0.47±0.08
0.51±0.05
0.45±0.05
0.0003
0.002
0.0002
0.0002
Vertical
Allocentric-Variability Control Experiment Slopes (comparison to one)
Horizontal
sv_sgs
sv_lgs
lv_sgs
lv_lgs
0.7±0.1
0.8±0.1
0.8±0.2
0.86±0.09
0.3
1.0
1.0
1.0
Vertical
sv_sgs
0.9±0.1
sv_lgs
1.0±0.2
lv_sgs
1.1±0.2
lv_lgs
1.0±0.1
Values are between-subjects means +/- standard error of the mean.
1.0
1.0
1.0
1.0
A
Reaching endpoint
Allocentric location
Original target
location
}
0
1
Allocentric Weight
B
Mean allocentric weights
revealing overall effect
of landmark shift
Mean allo weight
for upward-left shifts
{
Effect of landmark shift on reaching responses for one subject
(shift directions binned into four quadrants - one per panel)
6
Vertical target-relative reaching endpoint (cm)
4
Mean reaching
endpoint
2
0
Original target
-2
-4
Possible allocentric
locations
-6
6
4
2
0
-2
-4
-6
-6
-4
-2
0
2
4
6 -6
-4
-2
0
2
Horizontal target-relative reaching endpoint (cm)
4
6
Sample eye-movement and reaching data from
egocentric control experiment
Vertical screen-relative
eye position (cm)
10
Small gaze-shift
Large gaze-shift
5
0
-5
-10
-10
-5
0
5
-10
-5
0
5
10
3
6
Horizontal screen-relative eye position (cm)
Vertical target-relative
reaching endpoint (cm)
6
Small gaze-shift
Large gaze-shift
3
0
-3
-6
-6
-3
0
3
-6
-3
0
Horizontal target-relative reaching endpoint (cm)
Sample reaching data from the
allocentric control experiment
Small gaze-shift
Large gaze-shift
Large vibration
3
0
-3
-6
6
Small vibration
Vertical target-relative reaching endpoint (cm)
6
3
0
-3
-6
-6
-3
0
3
6 -6
-3
0
Horizontal target-relative reaching endpoint (cm)
3
6
Allocentric weights grouped by
landmark vibration amplitude
Small vibration
Large vibration
Allocentric weights grouped by
gaze shift amplitude
Small gaze shift
Large gaze shift
Allocentric Weight
0.8
0.7
Direct Fit
Full Model
Reduced Model
*
0.6
0.5
0.4
0.3
0.2
0.8
0.7
Small Vibration
Direct Fit
Full Model
Reduced Model
Large Vibration
*
0.6
0.5
0.4
0.3
0.2
Small Gaze Shift
Large Gaze Shift
A
10°
B
Up
0°
4s
6s
8s
20°
10s
10°
10°
Left
20°
20°
Right
10°
40°
Down
10°
Up
0°
D
4s
6s
8s
10s
10°
20°
20°
Down
20°
Up
Right
10°
0°
10°
20°
10°
40°
Down
10°
Up
0°
F
4s
6s
8s
10s
10°
20°
Down
20°
Up
10°
Left
20°
20°
Right
10°
0°
10°
20°
10°
30°
40°
Down
10°
Up
0°
H
4s
10°
6s
8s
10s
20°
Down
20°
Up
10°
Left
20°
20°
Right
10°
0°
10°
10°
30°
40°
20°
Left
30°
G
10°
10°
20°
E
0°
10°
30°
C
Up
Down
20°
Down
20°
Weights for each subject revealing
variability of shift effect
Allocentric weights grouped by landmark vibration amplitude
Small vibration
Direct Fit
Large vibration
Full Model
Allocentric weights grouped by gaze shift amplitude
Small gaze shift
Direct Fit
Large gaze shift
Full Model
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement