© 2005 Nature Publishing Group http://www.nature.com/natureneuroscience
ARTICLES
Prior experience of rotation is not required for recognizing objects seen from different angles Gang Wang1,2, Shinji Obama1, Wakayo Yamashita1, Tadashi Sugihara2 & Keiji Tanaka2 An object viewed from different angles can be recognized and distinguished from similar distractors after the viewer has had experience watching it rotate. It has been assumed that as an observer watches the rotation, separate representations of individual views become associated with one another. However, we show here that once monkeys learned to discriminate individual views of objects, they were able to recognize objects across rotations up to 60°, even though there had been no opportunity to learn the association between different views. Our results suggest that object recognition across small or medium changes in viewing angle depends on features common to similar views of objects.
An unfamiliar object often cannot be distinguished from similar or metrically varying distractors when the viewing angle changes1–6. Recognition across changes in viewing angle develops as the viewer sees the object while it rotates. Rotations provide two kinds of experience: exposure to different views of the object, and sequential pairing of these views with each other. The latter aspect has been central to discussions about the formation of the capability to recognize objects across changes in viewing angle. It has been assumed that different views of each object become associated with one another during object rotation, either through active learning or through passive experiencing of the successive appearance of nearby views7,8. This association, which forms during rotations, is thought to underlie object recognition ability across changes in viewing angle9–12. In contrast, the former aspect of observing an object in rotation— that is, the exposure to different views—has been relatively neglected. However, single-cell recordings in monkeys have shown that the neuronal representation of visual images changes as the monkey repeatedly experiences and discriminates between images13–18. Thus, the experience of different views alone, even if they do not appear in succession, may be important to the formation of perceptual tolerance to changes in viewing angle. We designed the present study to investigate the effects of experiencing different views without the experience of their successive appearance. A small proportion of cells in the anterior superior temporal sulcus and inferotemporal cortex of monkeys respond to all views of a particular person’s head or of a particular object, rather than to views of other people or objects19,20. Such perfect object selectivity across different views, a property of a small number of cells, might underlie the tolerance of perception to changes in viewing angle. In other studies, inferotemporal cells show moderately broad tunings for viewing angle of objects15,21–23. The response remains larger than half-maximum over viewing angles of 15–50° from the optimal. This partial object selectivity of a large number
of inferotemporal cells might also contribute to the tolerance of perception to changes in viewing angle. However, in these previous studies, the recordings were made after the monkeys had repeatedly experienced object rotations, and thus they do not address our present question: to what extent does object recognition across changes in viewing angle develop without the viewer’s experiencing rotation of an object? To address this issue, we conducted an experiment in which monkeys were exposed to different views of objects in an object recognition task. Monkeys, rather than humans, were used in the present study so that the behavioral results could later be compared with single-cell recording data in the same species. Similarity between objects and distractors affects the extent to which changes in viewing angle can be tolerated in the recognition of unfamiliar objects1–6,24–26; therefore, we systematically controlled the similarity between objects and distractors used in the experiments. We found that once monkeys learned to discriminate objects at each viewing angle, they were able to recognize objects across rotations up to 60°, even though there had been no opportunity to learn the association between different views. RESULTS For each experiment, we made four artificial objects by deforming a prototype in four different directions in a feature space, which was spanned by three combined parameters of the shape (Fig. 1a; also see Methods). Sets of objects were created by deforming individual prototypes to different extents (Fig. 1b.) Four different views of each object were created by rotating the object by 30° intervals around an axis perpendicular to the visual axis connecting the viewer’s eyes and the object. In each experiment, this set of 16 images was used to test object recognition across different viewing angles. Monkeys performed a discrimination task (‘Object task’) in which they had to detect a change in the identity of the object (Fig. 2). Two to five object images were presented sequentially in each trial: one to four
1Department
of Bioengineering, Faculty of Engineering, Kagoshima University, Kagoshima, Kagoshima 890-0065, Japan. 2Cognitive Brain Mapping Laboratory, RIKEN Brain Science Institute, Wako, Saitama 351-0198, Japan. Correspondence should be addressed to K.T. (
[email protected]). Received 19 April; accepted 21 October; published online 20 November 2005; doi:10.1038/nn1600
1568
VOLUME 8 | NUMBER 12 | DECEMBER 2005 NATURE NEUROSCIENCE
ARTICLES a
Object 4
Prototype
b
0
Version +8
o
30
o
o
60
o
90
Version 0
Prototype
Versio n
Object 3
Versio n
Object
© 2005 Nature Publishing Group http://www.nature.com/natureneuroscience
1 2 3
Object 2 0
4
+4
Object 1 Figure 1 Formation of object sets and example sets of stimulus images. (a) The four objects in a set were positioned at the vertices of a tetrahedron in a three-dimensional feature space. Seven parameters of the object shape were combined into three, which spanned the feature space. Object sets with various distances from the center (the ‘Prototype’) were made. On Version 0 human subjects showed ~80% correct performance. (b) The Prototype (top), sets of dissimilar (left) and similar objects (right). A set comprised 16 images, four views for each of four objects. Views are aligned horizontally, and objects are aligned vertically.
views of a first object were followed by one view of a second object. The monkey held down a lever while different views of the first object were presented. When an image of the second object appeared, the monkey had to release the lever. This task examined the monkeys’ capability to recognize objects while tolerating changes in viewing angle, as they had to distinguish between image changes resulting from changes in viewing angle alone and those resulting from changes in both viewing angle and the identity of object. Tests without prior experience of individual images In the first series of experiments, new stimulus sets were introduced in the object task while the knowledge of the task itself was maintained in the monkeys by using a consistent familiar set (Fig. 3a). Four different object sets with different levels of similarity were introduced to
Stimulus
on off
Fixation point
on off
Lever
press release
Eye fixation
inside outside
First object (1–4 times) 0.5 s 0.5 s
Second object
0.5 s 0.5 s
each monkey, one at a time. When the objects within the set were very different from one another, the monkeys showed good discrimination: the proportion of hits was significantly larger than that of false alarms (‘Version +8’ and ‘Version +4’ in Fig. 3b). They responded to object changes, but not to pure view changes. When the objects in the set were more similar to one another, both monkeys failed to discriminate object changes from pure view changes: there were no significant differences between the proportions of hits and false alarms at 30°, 60° and 90° rotations (Versions +2 and 0 in last three rows of Fig. 3b). These similar objects were still within the discrimination capability of the monkeys as, in post hoc tests, the monkeys could well discriminate the object images at each viewing angle (Fig. 3c). These findings, which are consistent with previous studies using human and monkey subjects1–6,24,25, gave the baseline performance without the prior experience either of individual views or of their sequential appearance.
0.5 s 0.5 s
0.5 s
0–1.0 s 0–1.5 s
Object task
First object
Preparatory task
Second object
First object
Second object
Figure 2 The time sequence of events in the task. The monkey had to hold down a lever while one to four views of a first object were presented and then release the lever when a view of a second object appeared. In the object task, but not in the preparatory task, the viewing angle of object images changed from presentation to presentation.
NATURE NEUROSCIENCE VOLUME 8 | NUMBER 12 | DECEMBER 2005
Tests with prior experience of individual images In the second series of experiments, we examined the monkeys’ capability to recognize objects across rotations after the monkeys had repeatedly experienced individual images of a new stimulus set in a preparatory task (Fig. 4a). In the preparatory task, the monkeys learned to discriminate images at each viewing angle, but there was no opportunity for them to learn the association between different views of each object. The training period lasted for at least 4 weeks before a new set was introduced to the object task. When new sets composed of similar objects (Fig. 4b) were introduced into the object task after the monkeys had experienced individual images, one monkey successfully discriminated object changes from pure view changes at 30° and 60° rotations, within the first 50 trials (Fig. 4c,
1569
a
Figure 3 Performance of monkeys immediately after the introduction of new stimulus sets into the object task, without prior experience of individual images. (a) The time course of the experiments. Each new object set was introduced while the monkey’s knowledge of the task was maintained by using a familiar set. The number within the box indicates the version number of the object set used in the experiment. (b) The white and black bars represent the hit and false alarm rates, respectively, at the second stimulus presentation in each trial. The values at the right (0°, 30°, 60° and 90°) indicate how much the viewing angle changed between the first and second presentations. Four sets of data, vertically aligned in a column, were obtained in one experiment. The number at the bottom indicates the version number of the object set used in the experiment. Smaller version numbers mean that objects within the set were more similar to one another. The data were pooled from the first 50 trials of each condition. *P < 0.01; **P < 0.001; ***P < 0.0001 (χ2 test). Note that the performance at 0° rotation did not require the tolerance of object recognition to changes in viewing angle. (c) The performance of monkeys in discriminating the object images within each viewing angle, examined in the preparatory task after the main test with the object task was completed. The hit and false alarm rates at the second presentation are plotted by the solid and broken lines, respectively. The x-axis represents the day after the start.
a
Monkey 1
New sets Familiar set
b
+4
0 +8 Object task Time
Monkey 1 1.0
***
*** **
***
*
0 1.0
Performance
0.5 0 1.0
***
*** ***
0 1.0
***
60°
*
0 Monkey 2 1.0
***
90°
Set 7, 0
0 1.0
***
0.5
*
0 Version +8
+4
Set 10, +2
0.5
+2
0
+8
+4
+2
0
0 1
Similarity
c
Monkey 1 *** ***
0 1.0 Performance
Set 1 series
Set 11 series
–2 +4 +2 0 Version
–2
***
Similarity
3 5 Day
7
The monkeys’ performance thus showed a clear contrast between the first and second series of experiments, for stimulus sets composed of similar objects. They successfully discriminated object changes from pure view changes at 30° and 60° rotations, when the test was conducted after they had experienced individual views of the objects (Fig. 4c); however, without prior experience of individual views, their performance was at chance level (Fig. 3b: Versions +2 and 0). Nevertheless, because different
Monkey 2 *** ***
0°
***
***
*** 30°
***
**
60°
0.5 0 1.0
***
0.5 0 1.0
90°
0.5 0 Set 2, 0 Set 8, 0
1570
Set 7, +2
0.5
Object task
0 1.0
+4 +2 0
***
0.5
0.5
0
0 1.0
30°
*
1.0
Set 8 series
0° *
Performance
***
1.0
0.5
Set 4, 0
0.5
Preparatory task Object task
Monkey 2
0.5 Set 2 series
Monkey 2 *** *
1.0
0.5 ***
Time Monkey 1
+2
0.5
New sets Familiar set
b
c
Object task
left). The second monkey also showed clear discrimination at 30° (Fig. 4c, right). At 60°, this monkey showed a significant difference between hit and false alarms in one experiment with one object set (‘Set 1, 0’), but not in the other experiment conducted with another set (‘Set 11, 0’). Neither monkey showed discrimination at 90°. The discrimination at 90° developed within a week after the introduction of the new stimulus sets (Supplementary Fig. 1 online).
Performance
© 2005 Nature Publishing Group http://www.nature.com/natureneuroscience
ARTICLES
Set 1, 0 Set 11, 0
Figure 4 Performance when new stimulus sets were introduced into the object task after each monkey had discriminated the images from one another at each viewing angle in the preparatory task. (a) The time course of the experiments on each monkey. The same procedure was repeated twice with different object sets. (b) The performance of monkeys with the object set series in the preparatory task. The hit and false alarm rates at the second presentation are represented by the solid and broken lines, respectively. The x-axis represents the version number of stimulus set. Smaller version numbers indicate that objects were more similar to one another in the set. The vertical broken lines indicate the versions used later in the object task. (c) Performance on the initial 50 trials of each condition in the object task (notations as in Fig. 3b). The similarity between objects within the sets used in these tests was comparable to that of the most similar sets used in the test without prior experience of individual images (Version 0; see also Supplementary Fig. 2).
VOLUME 8 | NUMBER 12 | DECEMBER 2005 NATURE NEUROSCIENCE
ARTICLES a
Preparatory task
Object task
Monkey 1 Set 9, +2 Familiar set Object task
Set 12, 0
Monkey 2 Set 9, +2
Set 12, 0 Familiar set Object task
Set 9 series
c
Set 12 series
Set 9, +2 1.0
Set 12, 0 ***
0° **
0.5
1.0
0 1.0
0.5
***
0
+4 +2 0 –2 +4 +2 0 –2 Version Version Monkey 2
Performance
Performance
Monkey 1
*** 30°
0.5 0 1.0
**
60°
0.5
Performance
prior experience, but on different monkeys (cross-design). For each set, only one mon90° key was given prior experience discriminating 0.5 individual images at each viewing angle in the 0.5 0 preparatory task (Fig. 5a). The object sets used +4 +2 0 –2 +4 +2 0 –2 0 in this test were composed of similar objects Version Version Without With Without With (Fig. 5b). With prior experience of individual Monkey 2 Monkey 1 Monkey 1 Monkey 2 images, the monkeys discriminated object changes from pure view changes of up to 60° or object sets were used in the two series of experiments, a small possibil- 30° when new sets were introduced into the object task (columns marked ity remained that the differences were due to unexpected differences in ‘With’ in Fig. 5c). There was no sign of discrimination ability without prior experience (columns marked ‘Without’ in Fig. 5c). This contrast of stimulus objects rather than to differences in prior experience. To exclude this possibility, we conducted an additional series of tests, performance with the same object set confirmed that the differences in in which the same object sets were used for the tests with and without performance were due to prior experience of individual images. 1.0
0 1.0
1
Set 1, 0
Set 2, 0
Set 8, 0
Set 11, 0
0 1
Distance
© 2005 Nature Publishing Group http://www.nature.com/natureneuroscience
Time
b
Figure 5 Performance of monkeys in cross-design tests. The notations are the same as in Figures 3 and 4. (a) The performance on Set 9, +2 was first tested on Monkey 1 with prior experience of individual images in the preparatory task, and then on Monkey 2 without prior experience. Conversely, the performance on Set 12, 0 was first tested on Monkey 2 with prior experience, and then on Monkey 1 without prior experience. (b) Performance of monkeys on the Set 9 series (left) and Set 12 series (right) in the preparatory task. Two sets of data (Set 9 series in Monkey 1 and Set 12 series in Monkey 2) were obtained before the object test was conducted with the stimulus set, whereas the other two sets (Set 9 series in Monkey 2 and Set 12 series in Monkey 1) were obtained after the object test. (c) Initial performance of monkeys in the object task.
0 1
Set 12, 0
Set 9, +2 0
Control analyses and experiments One may suppose that the discrimination of object changes from pure view changes was due to differences in similarity between pairs of views within objects and pairs across objects. This was not the case in the stimulus sets we used to examine the effects of prior experience of individual images (sets of Versions 0 and +2). We calculated the similarity between two images as the Euclidean distance between coefficients of their wavelet transformations. The position and orientation of one image relative to the other was changed to obtain the minimum distance. When the distances were compared between pairs within objects and pairs across objects with the same differences in viewing angle, the two distributions corresponded well with each other at viewing-angle differences of 30–90° (Fig. 6); further, there were no significant differences in any comparison in any stimulus set used to test the effects of prior experience of individual images (P > 0.05, t-test). The same results were obtained when the similarity between two images was calculated as the Euclidean distance between the luminosity values in individual pixels with adjustment of position and orientation. Therefore, the monkeys’ discrimination of object changes from pure view changes did not depend on the general similarity of images.
1
Set 13, 0
Set 14, 0
0 0° 10° 20° 30° 60° 90°
0° 10° 20° 30° 60° 90°
Difference in viewing angle
NATURE NEUROSCIENCE VOLUME 8 | NUMBER 12 | DECEMBER 2005
Figure 6 Similarity between images for within-object and acrossobject pairs. The euclidean distance between coefficients of wavelet transformations of the images was calculated, normalized by the maximum distance in each set and plotted against the difference in viewing angle between the images. Dotted lines represent pairs within objects and solid lines represent pairs across objects. Note that only views with 30°, 60° and 90° intervals were used in the present study. Error bars represent s.d.
1571
a
b
α β γ δ
11
12
13
14
21
22
23
24
31
32
33
34
41
42
43
44
Pseudo association
Preparatory task
β
New sets
γ
Familiar set
Object task time
δ
0°
α
30° 60°
c
Set 3 series
d
1.0
Monkey 1
1.0
90° Set 3, +1
0.5 0.5 0 0 1.0
1 Set 6 series
0.5
Performance
Figure 7 Control experiments with pseudo combinations of object views. (a) Different views of different objects were combined. The first digits indicate the object number and the suffixes indicate the view number. The pseudo combinations are indicated by lines and marked by the Greek letters. The monkeys were required to neglect changes within the pseudo combinations and respond to other changes. (b) The time course of the experiments on Monkey 1. The test of pseudo association started after the monkey experienced individual images in the preparatory task. The same procedure was repeated twice with different stimulus sets on Monkey 1, whereas only one experiment was conducted on Monkey 2. (c) The performance of monkeys with the object set series in the preparatory task. (d) Daily performance of monkeys in the pseudo association task. The hit and false alarm rates at the second presentation in each trial are represented by the solid and broken lines, respectively. The performance at 0°, 30°, 60° and 90° changes from the first to the second presentation is indicated by the gray, red, green and blue lines, respectively. The x-axis represents the day after the start.
Performance
3
2
4
5
7
6
1.0 Set 6, +4
0.5 0
0 1.0
1
2
Set 2 series
3
5
4
Monkey 2
Set 2, +2
1.0
Performance
It is also possible that the monkeys learned the association between pairs of views of each 0.5 0.5 object in the first 50 trials after the introduction of a new set. To examine this possibility, we 0 0 artificially grouped object images across objects +4 +2 0 –2 1 3 5 2 4 (‘pseudo combinations’, Fig. 7a); we then Version Day trained the monkeys to hold the lever down when the image changed to another member of the pseudo combination and to release it when the image changed to the first appearance of within-object pairs (Fig. 8b), indicating that the one belonging to other combinations. New stimulus sets were introduced monkeys discriminated object changes from pure view changes at the to this pseudo association task after the monkeys had experienced indi- first appearance of each pair. To infer what visual cues corresponded to the changes in performance vidual images in the preparatory task (Fig. 7b). The stimulus sets used in this test were composed of similar objects (Fig. 7c). If the monkeys’ between a 30° rotation and a 90° rotation or between a 60° rotation and successful performance in the object task had been achieved by a quick a 90° rotation, we analyzed feature differences between views of each learning of associations between views, the pseudo associations should also have been learned quickly. We found that the monkeys completely failed to learn this task—not only within the first day, but also over a Monkey 2 Monkey 1 week (Fig. 7d). a To examine whether the monkeys discriminated object changes from 1.0 pure view changes from the beginning of the test with the object task, we also conducted an experiment in which we measured each monkey’s performance at the first appearance of each pair of views. New sets composed 0.5 of similar objects were introduced to the object task after the experience of individual images in the preparatory task (Fig. 8a). The change in viewing angle between successively presented stimuli was limited to 30°, and the stimulus sequences were controlled so that new pairs always 0 +4 +2 0 –2 +4 +2 0 –2 appeared as the first and second stimuli in the sequence within a trial. In Set 13 series both monkeys, the proportion of correct releases at the first appearance Set 14 series of across-object pairs was significantly larger than that of false releases at b 1.0 Figure 8 Performance on the first appearance of each pair of object views. Notations are the same as in Figures 3–5. (a) The monkeys’ performance with the object set series in the preparatory task. (b) The hit and false alarm rates on the first appearance of each pair of images. We controlled the stimulus sequence so that new pairs always appeared as the first and second stimuli in the sequence of a trial.
1572
Performance
© 2005 Nature Publishing Group http://www.nature.com/natureneuroscience
ARTICLES
***
30° **
0.5
0
Set 13, 0
Set 14, 0
VOLUME 8 | NUMBER 12 | DECEMBER 2005 NATURE NEUROSCIENCE
© 2005 Nature Publishing Group http://www.nature.com/natureneuroscience
ARTICLES object in the stimulus sets used to examine the effects of prior experience of individual images. Because we did not know which features the monkeys used to perform the task, we arbitrarily focused on the seven shape parameters that we had systematically changed to create four objects from the prototype (see Methods). We measured the value of each parameter in the projected images, normalized the values so that the largest and smallest values of each parameter became 1 and 0, respectively, and calculated absolute differences of normalized values of each projected parameter between views of each object. The differences were, on average, 0.18 for view pairs with 30° difference in viewing angle, 0.32 for 60° pairs and 0.49 for 90° pairs. The large value for 90° pairs means that prominent differences in some parameters between objects at one viewing angle tended to disappear or change directions with 90° rotations. Therefore, features that varied with the parameters were useless in pairing views of each object across 90° changes in viewing angle. There were no such drastic differences for view pairs with 30° or 60° separation in our stimulus sets. DISCUSSION Our data indicate that the monkeys discriminated object changes from pure view changes without any prior experience of the pairing between views of each object. Repeatedly experiencing individual images and discriminating among these images at each viewing angle was necessary and sufficient for the formation of this capability. However, this was true only for viewing-angle changes up to 60°. Object recognition across viewpoint changes >60° required specific learning of the pairing. This finding applies only to object recognition among similar or metrically varying objects. When distractors were very different from the target object, recognition across views did not require prior experience of the objects at all1,4,5,24,26. One concern is that the difference in performance between the tests with and without prior experience of individual images was due to unexpected differences in the object similarity between the object sets used in the two types of tests. This possibility was excluded for the following three reasons. First, the amount of deformation used in the monkey experiments had previously been normalized by using human psychophysics (see Methods). Second, we confirmed, in the monkeys, that the object sets used for the tests without prior experience were not more difficult to discriminate than those used for the test with prior experience. For the stimuli used in the test with prior experience, the threshold of discrimination had been determined in the monkeys in the preparatory task, and we used the sets just above the discrimination threshold (Fig. 4b). For the stimuli used in the test without prior experience, we ran the preparatory task with the set after the test with object task was completed, thus confirming that the set was within the monkeys’ capability of discrimination (Fig. 3c). Third, the cross-design series of tests showed a clear contrast of performance on the same object sets with and without prior experience (Fig. 5c). The formation of object recognition across viewpoints without any prior experience of pairing between views of each object was not specific to particular object shapes, because similar results were obtained with eight different sets of objects (Supplementary Fig. 2 online). In addition, by analyzing the images, we excluded the possibility that the monkeys depended on differences in general image similarity between pairs of views within objects and pairs across objects. In the tested range of viewing-angle differences (30–90°), the similarity between images was comparable between pairs within objects and pairs across objects for all the stimulus sets we used to examine the effects of prior experience of individual views (Fig. 6). Previous single-cell recording studies have shown that as a monkey learns to discriminate among stimulus images, the number of cells that are responsive to these images increases in the inferotemporal cortex15,16.
NATURE NEUROSCIENCE VOLUME 8 | NUMBER 12 | DECEMBER 2005
Responses of inferotemporal cells become more selective for stimulus shape parameters that are useful for discrimination than for irrelevant parameters17,18. In the present study, during the prior experience in the preparatory task, the monkeys learned to discriminate between images at each of several viewing angles, but they did not have any opportunity to learn the pairing between different views of each object. The monkeys’ ability to recognize objects across views, immediately after the introduction of the stimulus sets into the object task, suggests that the features for which inferotemporal cells became tuned during the image discrimination were common to nearby views. Because the emergent representations of nearby views of each object had common features, the monkeys recognized the objects even across views when they were introduced to the object task. Alternatively, feature-based attention might have developed during the image discrimination. The features to which attention was directed for the detection of object changes within each viewing angle were common to nearby views. In either case, the monkeys discriminated features that did not change with rotation from view-dependent features that changed significantly with rotation. Cells in the monkey inferotemporal cortex tend to be more sensitive to rotation-independent types of changes in object images than to changes that occur as a result of rotation27,28. Monkey inferotemporal cells tend to respond more similarly to different views of same objects than to different views of different objects (T.S., S. Edelman & K.T., Soc. Neurosci. Abstr. 26, 2000). This tendency exists not only for objects for which the monkey has learned the pairing between views, but also for objects for which the monkey has only experienced individual views separately. Features that are common to nearby views may be preferentially used by inferotemporal cells to represent object images, regardless of prior experience of the pairing between views of each object. As the difference in viewing angle increases between two views of an object, there tend to be fewer features in common across the two views. In the object sets of Versions 0 and +2, with which the monkeys showed different performances depending on prior experience of individual views, common features largely disappeared as the viewing-angle difference approached 90°. This fact is consistent with the monkeys’ successful recognition of objects at viewing angles of 60° but not 90°. However, depending on the object shapes and view-range selections, discriminative features may disappear or lose the correspondence in parametric rank order even with smaller changes in viewing angle. Therefore, the precise viewing-angle ranges over which object recognition was retained in the current study cannot be immediately generalized to other cases or other object sets. In conclusion, the present results suggest that the tolerance of object recognition to changes in viewing angle of up to medium magnitude (~60° for the objects used in the present study) has a different mechanism than the tolerance to larger changes in viewing angle. Regularities existing between nearby views of each object underlie object recognition for medium changes in viewing angle, whereas specific learning of pairing between views of each object is necessary for object recognition across larger changes in viewing angle. METHODS We used two male macaque monkeys (Macaca fuscata) weighing 9 kg and 10 kg, respectively. In a preparatory surgery, we implanted a titanium head holder in the skull with titanium screws. All procedures on the monkeys were performed in accordance with the guidelines of the Japan Neuroscience Society and were approved by the Animal Experiment Committee of Kagoshima University. Stimulus objects. Stimulus objects were created using three-dimensional (3D) graphic software (Shade 6, e-frontier). A prototype was deformed in four directions in a 3D feature space to make four daughter objects. We used human psychophysics to adjust the directions of deformations and the relative amount of
1573
© 2005 Nature Publishing Group http://www.nature.com/natureneuroscience
ARTICLES deformation in the four directions so that the positions of the daughter objects formed a tetrahedron with sides of equal lengths in psychological space (Fig. 1a). We also used human psychophysics to calibrate the amount of deformation in different object set series. The presentation condition in the human experiments was adjusted so that the percentages of correct responses roughly corresponded to those of the monkeys in the preparatory task. We defined several related object sets: Version 0 was the set yielding ~80% correct performance. The amount of deformation in Version 0 was multiplied by 0.4, 0.7, 1.3, 1.6, 1.9, 2.2 and 3.4 to make objects for Versions –2, –1, 1, 2, 3, 4 and 8, respectively. Because seven shape parameters were combined into three to span the 3D space and because none of six sides of the tetrahedron formed by the four objects was parallel to the axes of the feature space, differences between daughter objects were distributed among the seven parameters. We generated 14 series of object sets from 14 prototypes, which were quite different from one another (Supplementary Fig. 3 online). Object sets made from different prototypes were used in different experiments to prevent the generalization of learning from one experiment to another. The sizes of object images were 6.5° on average. Procedure. We formed a 3D shape by combining several geometrical primitives, by deforming some of the primitives, or both. We then selected seven parameters of the shape (for example, length, diameter and curvature of each part, and angle between two parts). Some of these parameters were combined by making the sum or ratio of two measures a constant, so as to reduce the number of parameters to three. We then determined the range for each parameter that did not provide any abrupt changes in shape. The three parameters were normalized so that a given amount of numerical change in each of the three parameters produced roughly the same perceptual changes. Next, four positions were selected in the 3D feature space spanned by the three parameters so that they formed a tetrahedron. None of six sides of the tetrahedron was parallel to the axes of the feature space. The prototype was determined by the geometrical center of the four positions. Many different objects were defined by positions along the lines connecting the prototype and the original four object positions. We then conducted human psychophysics with the original set of four objects, using a delayed match-to-sample task. On each trial, we presented an alert signal (500 ms), which was followed by a sample at the center (200 ms), a mask (500 ms), a target and distractor side by side (500 ms) and another mask (500 ms). The target was a different view of the object presented as the sample. The distractor was a view of another object from the same set. The subject reported the side of the target (left or right) by pushing one of two buttons within 1,000 ms from the onset of the target and distractor. There was a total of 120 trials, 20 trials for each object combination. If the percentage of correct responses was >80% for all three tests using one object, then that object was replaced with a new object located at a position 20% closer to the prototype along the same deformation axis. If the percentage of correct responses was
