The perception of emotion and social cues in faces.
Explore the source record for details and available documents.
Cues↗
Biomedical subjects
James V Haxby
Publications and source records attributed to James V Haxby.
Neural response to the visual familiarity of faces.
Recognizing personally familiar faces is the result of a spatially distributed process that involves visual perceptual areas and areas that play a role in other cognitive and social functions, such as the anterior paracingulate cortex, the precuneus and the amygdala [M.I. Gobbini, E. Leibenluft, N. Santiago, J.V. Haxby, Social and emotional attachment in the neural representation of faces, Neuroimage 22 (2004) 1628-1635; M.I. Gobbini, J.V. Haxby, Neural systems for recognition of familiar faces, Neuropsychologia, in press; E. Leibenluft, M.I. Gobbini, T. Harrison, J.V. Haxby, Mothers' neural activation in response to pictures of their, and other, children, Biol. Psychiatry 56 (2004) 225-232]. In order to isolate the role of visual familiarity in face recognition, we used fMRI to measure the response to faces characterized by experimentally induced visual familiarity that carried no biographical information or emotional content. The fMRI results showed a stronger response in the precuneus to the visually familiar faces consistent with studies that implicate this region in the retrieval of information from long-term memory and imagery. Moreover, this finding supports the hypothesis of a key role for the precuneus in the acquisition of familiarity with faces [H. Kosaka, M. Omori, T. Iidaka, T. Murata, T. Shimoyama, T. Okada, N. Sadato, Y. Yonekura, Y. Wada, Neural substrates participating in acquisition of facial familiarity: an fMRI study, Neuroimage 20 (2003) 1734-1742]. By contrast, the visually familiar faces evoked a weaker response in the fusiform gyrus, which may reflect the development of a sparser encoding or a reduced attentional load when processing stimuli that are familiar. The visually familiar faces also evoked a weaker response in the amygdala, supporting the proposed role of this structure in mediating the guarded attitude when meeting someone new.
Adult↗
Beyond mind-reading: multi-voxel pattern analysis of fMRI data.
A key challenge for cognitive neuroscience is determining how mental representations map onto patterns of neural activity. Recently, researchers have started to address this question by applying sophisticated pattern-classification algorithms to distributed (multi-voxel) patterns of functional MRI data, with the goal of decoding the information that is represented in the subject's brain at a particular point in time. This multi-voxel pattern analysis (MVPA) approach has led to several impressive feats of mind reading. More importantly, MVPA methods constitute a useful new tool for advancing our understanding of neural information processing. We review how researchers are using MVPA methods to characterize neural coding and information processing in domains ranging from visual perception to memory search.
Algorithms↗
Neural systems for recognition of familiar faces.
Immediate access to information about people that we encounter is an essential requirement for effective social interactions. In this manuscript we briefly review our work and work of others on familiar face recognition and propose a modified version of our model of neural systems for face perception with a special emphasis on processes associated with recognition of familiar faces. We argue that visual appearance is only one component of successful recognition of familiar individuals. Other fundamental aspects include the retrieval of "person knowledge" - the representation of the personal traits, intentions, and outlook of someone we know - and the emotional response we experience when seeing a familiar individual. Specifically, we hypothesize that the "theory of mind" areas, that have been implicated in social and cognitive functions other than face perception, play an essential role in the spontaneous activation of person knowledge associated with the recognition of familiar individuals. The amygdala and the insula, structures that are involved in the representation of emotion, also are part of the distributed network of areas that are modulated by familiarity, reflecting the role of emotion in face recognition.
Emotions↗
Spontaneous retrieval of affective person knowledge in face perception.
In a functional magnetic resonance imaging experiment, we explored whether affective person knowledge based on memories formed from minimal information is spontaneously retrieved in face perception. In the first stage of the experiment, participants were presented with 120 unfamiliar faces. Each face was presented with a description of one of four types of behaviors: aggressive, disgusting, neutral, and nice. In the second stage, participants were scanned while engaged in a one-back recognition task in which they saw the faces that were associated with behaviors and 30 novel faces. Although this task is a simple perceptual task that neither demands person evaluation nor retrieval of person knowledge, neural responses to faces differed as a function of the behaviors. Faces associated with behaviors evoked stronger activity than did novel faces in regions implicated in social cognition--anterior paracingulate cortex and superior temporal sulcus. Explicit memory for the behaviors enhanced the neural response in these regions. Faces associated with disgusting behaviors evoked stronger activity in left anterior insula than did faces associated with aggressive behaviors. This effect was equally strong for faces associated with explicitly recalled behaviors and faces associated with non-recalled behaviors. The findings suggest that affective person knowledge acquired from minimal information is spontaneously retrieved in face perception, engaging neural systems for analysis of social cognition and emotions.
Adolescent↗
Dissociation of face-selective cortical responses by attention.
We studied attentional modulation of cortical processing of faces and houses with functional MRI and magnetoencephalography (MEG). MEG detected an early, transient face-selective response. Directing attention to houses in "double-exposure" pictures of superimposed faces and houses strongly suppressed the characteristic, face-selective functional MRI response in the fusiform gyrus. By contrast, attention had no effect on the M170, the early, face-selective response detected with MEG. Late (>190 ms) category-related MEG responses elicited by faces and houses, however, were strongly modulated by attention. These results indicate that hemodynamic and electrophysiological measures of face-selective cortical processing complement each other. The hemodynamic signals reflect primarily late responses that can be modulated by feedback connections. By contrast, the early, face-specific M170 that was not modulated by attention likely reflects a rapid, feed-forward phase of face-selective processing.
Attention↗
Fine structure in representations of faces and objects.
Explore the source record for details and available documents.
Animals↗
Partially distributed representations of objects and faces in ventral temporal cortex.
Object and face representations in ventral temporal (VT) cortex were investigated by combining object confusability data from a computational model of object classification with neural response confusability data from a functional neuroimaging experiment. A pattern-based classification algorithm learned to categorize individual brain maps according to the object category being viewed by the subject. An identical algorithm learned to classify an image-based, view-dependent representation of the stimuli. High correlations were found between the confusability of object categories and the confusability of brain activity maps. This occurred even with the inclusion of multiple views of objects, and when the object classification model was tested with high spatial frequency "line drawings" of the stimuli. Consistent with a distributed representation of objects in VT cortex, the data indicate that object categories with shared image-based attributes have shared neural structure.
Algorithms↗
Sustained attention in mild Alzheimer's disease.
The vigilance decrement in perceptual sensitivity was examined in 10 patients with mild Alzheimer's disease (AD) and 20 age-matched controls. A visual high-event rate digit-discrimination task lasting 7.2 min. (six 1.2 min blocks) was presented at different levels of stimulus degradation. Previous studies have shown that sensitivity decrements (d') over time at high-stimulus degradation result from demands on effortful processing. For all degradation levels, the overall level of vigilance (d') was lower in AD patients than in controls. All participants showed sensitivity decrement over blocks, with greater decrement at higher degradation levels. AD patients exhibited greater sensitivity decrement over time at the highest degradation level they all could perform relative to control participants. There were no concomitant changes in either response bias (C) or response times. The results indicate that mild AD patients have overall lower levels of vigilance under conditions that require both automatic and effortful processing. Mild AD patients also exhibit a deficit in the maintenance of vigilance over time under effortful processing conditions. Although the sample of AD patients was small, results further suggest that both possible and probable AD patients had greater sensitivity decrement over time at the highest degradation level than did control participants, but only probable AD patients had lower overall levels of vigilance. In the possible AD patients as a group, the decrement in vigilance occurred in the absence of concurrent deficits on standard attentional tasks, such as the Stroop and Trail Making tests, suggesting that deficits in vigilance over time may appear earlier than deficits in selective attention.
Aged↗
Mothers' neural activation in response to pictures of their children and other children.
BACKGROUND: Considerable literature has focused on neural responses evoked by face viewing. We extend that literature and explore the neural correlates of maternal attachment with an fMRI study in which mothers view photographs of their own children. METHOD: Seven mothers performed a one-back repetition detection task while viewing photographs of their own child, friends of their child, unfamiliar children, and unfamiliar adults. RESULTS: Viewing one's own child versus a familiar child was associated with activation in the amygdala, insula, anterior paracingulate cortex, and posterior superior temporal sulcus (STS). Viewing familiar versus unfamiliar children elicited increased activation in regions associated with familiarity in adults. Viewing unfamiliar children versus unfamiliar adults was associated with activation in the fusiform gyrus, intraparietal sulcus, precuneus, and posterior STS. CONCLUSIONS: The sight of one's own child versus that of a familiar child activates regions that mediate emotional responses (amygdala, insula) and are associated with theory of mind functions (anterior paracingulate cortex, posterior superior temporal sulcus). These activations may reflect the intense attachment, vigilant protectiveness, and empathy that characterize normal maternal attachment. The sight of an unfamiliar child's face compared with that of an unfamiliar adult engages areas associated with attention as well as face perception.
Adult↗
Beyond sensory images: Object-based representation in the human ventral pathway.
We investigated whether the topographically organized, category-related patterns of neural response in the ventral visual pathway are a representation of sensory images or a more abstract representation of object form that is not dependent on sensory modality. We used functional MRI to measure patterns of response evoked during visual and tactile recognition of faces and manmade objects in sighted subjects and during tactile recognition in blind subjects. Results showed that visual and tactile recognition evoked category-related patterns of response in a ventral extrastriate visual area in the inferior temporal gyrus that were correlated across modality for manmade objects. Blind subjects also demonstrated category-related patterns of response in this "visual" area, and in more ventral cortical regions in the fusiform gyrus, indicating that these patterns are not due to visual imagery and, furthermore, that visual experience is not necessary for category-related representations to develop in these cortices. These results demonstrate that the representation of objects in the ventral visual pathway is not simply a representation of visual images but, rather, is a representation of more abstract features of object form.
Adult↗
Social and emotional attachment in the neural representation of faces.
To dissociate the role of visual familiarity from the role of social and emotional factors in recognizing familiar individuals, we measured neural activity using functional magnetic resonance imaging (fMRI) while subjects viewed (1) faces of personally familiar individuals (i.e. friends and family), (2) faces of famous individuals, and (3) faces of strangers. Personally familiar faces evoked a stronger response than did famous familiar faces and unfamiliar faces in areas that have been associated with 'theory of mind', and a weaker response in the amygdala. These response modulations may reflect the spontaneous activation of social knowledge about the personality and attitudes of close friends and relatives and the less guarded attitude one has around these people. These results suggest that familiarity causes changes in neural response that extend beyond a visual memory for a face.
Adult↗
Combinatorial codes in ventral temporal lobe for object recognition: Haxby (2001) revisited: is there a "face" area?
Haxby et al. [Science 293 (2001) 2425] recently argued that category-related responses in the ventral temporal (VT) lobe during visual object identification were overlapping and distributed in topography. This observation contrasts with prevailing views that object codes are focal and localized to specific areas such as the fusiform and parahippocampal gyri. We provide a critical test of Haxby's hypothesis using a neural network (NN) classifier that can detect more general topographic representations and achieves 83% correct generalization performance on patterns of voxel responses in out-of-sample tests. Using voxel-wise sensitivity analysis we show that substantially the same VT lobe voxels contribute to the classification of all object categories, suggesting the code is combinatorial. Moreover, we found no evidence for local single category representations. The neural network representations of the voxel codes were sensitive to both category and superordinate level features that were only available implicitly in the object categories.
Adult↗
FMRI responses to video and point-light displays of moving humans and manipulable objects.
We used fMRI to study the organization of brain responses to different types of complex visual motion. In a rapid event-related design, subjects viewed video clips of humans performing different whole-body motions, video clips of manmade manipulable objects (tools) moving with their characteristic natural motion, point-light displays of human whole-body motion, and point-light displays of manipulable objects. The lateral temporal cortex showed strong responses to both moving videos and moving point-light displays, supporting the hypothesis that the lateral temporal cortex is the cortical locus for processing complex visual motion. Within the lateral temporal cortex, we observed segregated responses to different types of motion. The superior temporal sulcus (STS) responded strongly to human videos and human point-light displays, while the middle temporal gyrus (MTG) and the inferior temporal sulcus responded strongly to tool videos and tool point-light displays. In the ventral temporal cortex, the lateral fusiform responded more to human videos than to any other stimulus category while the medial fusiform preferred tool videos. The relatively weak responses observed to point-light displays in the ventral temporal cortex suggests that form, color, and texture (present in video but not point-light displays) are the main contributors to ventral temporal activity. In contrast, in the lateral temporal cortex, the MTG responded as strongly to point-light displays as to videos, suggesting that motion is the key determinant of response in the MTG. Whereas the STS responded strongly to point-light displays, it showed an even larger response to video displays, suggesting that the STS integrates form, color, and motion information.
Adult↗
Parallel visual motion processing streams for manipulable objects and human movements.
We tested the hypothesis that different regions of lateral temporal cortex are specialized for processing different types of visual motion by studying the cortical responses to moving gratings and to humans and manipulable objects (tools and utensils) that were either stationary or moving with natural or artificially generated motions. Segregated responses to human and tool stimuli were observed in both ventral and lateral regions of posterior temporal cortex. Relative to ventral cortex, lateral temporal cortex showed a larger response for moving compared with static humans and tools. Superior temporal cortex preferred human motion, and middle temporal gyrus preferred tool motion. A greater response was observed in STS to articulated compared with unarticulated human motion. Specificity for different types of complex motion (in combination with visual form) may be an organizing principle in lateral temporal cortex.
Brain Mapping↗
Human neural systems for face recognition and social communication.
Face perception is mediated by a distributed neural system in humans that consists of multiple, bilateral regions. The functional organization of this system embodies a distinction between the representation of invariant aspects of faces, which is the basis for recognizing individuals, and the representation of changeable aspects, such as eye gaze, expression, and lip movement, which underlies the perception of information that facilitates social communication. The system also has a hierarchical organization. A core system, consisting of occipitotemporal regions in extrastriate visual cortex, mediates the visual analysis of faces. An extended system consists of regions from neural systems for other cognitive functions that can act in concert with the core system to extract meaning from faces. Of regions in the extended system for face perception, the amygdala plays a central role in processing the social relevance of information gleaned from faces, particularly when that information may signal a potential threat.
Attention↗
Visual imagery of famous faces: effects of memory and attention revealed by fMRI.
Complex pictorial information can be represented and retrieved from memory as mental visual images. Functional brain imaging studies have shown that visual perception and visual imagery share common neural substrates. The type of memory (short- or long-term) that mediates the generation of mental images, however, has not been addressed previously. The purpose of this study was to investigate the neural correlates underlying imagery generated from short- and long-term memory (STM and LTM). We used famous faces to localize the visual response during perception and to compare the responses during visual imagery generated from STM (subjects memorized specific pictures of celebrities before the imagery task) and imagery from LTM (subjects imagined famous faces without seeing specific pictures during the experimental session). We found that visual perception of famous faces activated the inferior occipital gyri, lateral fusiform gyri, the superior temporal sulcus, and the amygdala. Small subsets of these face-selective regions were activated during imagery. Additionally, visual imagery of famous faces activated a network of regions composed of bilateral calcarine, hippocampus, precuneus, intraparietal sulcus (IPS), and the inferior frontal gyrus (IFG). In all these regions, imagery generated from STM evoked more activation than imagery from LTM. Regardless of memory type, focusing attention on features of the imagined faces (e.g., eyes, lips, or nose) resulted in increased activation in the right IPS and right IFG. Our results suggest differential effects of memory and attention during the generation and maintenance of mental images of faces.
Adult↗