Biomedical subjects

Mikko Sams

Publications and source records attributed to Mikko Sams.

28 records · Page 2Linked to original sources

PubMed2004

Factors influencing audiovisual fission and fusion illusions.

Information processing in auditory and visual modalities interacts in many circumstances. Spatially and temporally coincident acoustic and visual information are often bound together to form multisensory percepts [B.E. Stein, M.A. Meredith, The Merging of the Senses, A Bradford Book, Cambridge, MA, (1993), 211 pp.; Psychol. Bull. 88 (1980) 638]. Shams et al. recently reported a multisensory fission illusion where a single flash is perceived as two flashes when two rapid tone beeps are presented concurrently [Nature 408 (2000) 788; Cogn. Brain Res. 14 (2002) 147]. The absence of a fusion illusion, where two flashes would fuse to one when accompanied by one beep, indicated a perceptual rather than cognitive nature of the illusion. Here we report both fusion and fission illusions using stimuli very similar to those used by Shams et al. By instructing subjects to count beeps rather than flashes and decreasing the sound intensity to near threshold, we also created a corresponding visually induced auditory illusion. We discuss our results in light of four hypotheses of multisensory integration, each advocating a condition for modality dominance. According to the discontinuity hypothesis [Cogn. Brain Res. 14 (2002) 147], the modality in which stimulation is discontinuous dominates. The modality appropriateness hypothesis [Psychol. Bull. 88 (1980) 638] states that the modality more appropriate for the task at hand dominates. The information reliability hypothesis [J.-L. Schwartz, J. Robert-Ribes, P. Escudier, Ten years after Summerfield: a taxonomy of models for audio-visual fusion in speech perception. In: R. Campbell (Ed.), Hearing by Eye: The Psychology of Lipreading, Lawrence Earlbaum Associates, Hove, UK, (1998), pp. 3-51] claims that the modality providing more reliable information dominates. In strong forms, none of these three hypotheses applies to our data. We re-state the hypotheses in weak forms so that discontinuity, modality appropriateness and information reliability are factors which increase a modality's tendency to dominate. All these factors are important in explaining our data. Finally, we interpret the effect of instructions in light of the directed attention hypothesis which states that the attended modality is dominant [Psychol. Bull. 88 (1980) 638].

Acoustic Stimulation↗

PubMed2003

Integration of heard and seen speech: a factor in learning disabilities in children.

Normal-learning children (NL) and children with learning disabilities (LD) reported their perceptions of unisensory (auditory or visual), concordant audiovisual (e.g. visual /apa/ and auditory /apa/) and conflicting (e.g. visual /aka/ and auditory /apa/) speech stimuli in quiet and noise (0 dB and -12 dB signal-to-noise ratio, SNR). In normal populations, watching such conflicting combinations typically changes auditory percepts ('McGurk effect'). NL and LD children identified unisensory auditory and congruent audiovisual stimuli similarly in all conditions. Despite being less accurate identifying unisensory visual stimuli, LD children were more likely than NL children to report hearing only the visual component of incongruent audiovisual stimuli at -12 dB SNR. Furthermore, LD children with brainstem timing deficits demonstrated a distinctive pattern of audiovisual perception. The results suggest that the perception of simultaneous auditory and visual speech differs between NL and LD children, perhaps reflecting variations in neural processing underlying multisensory integration.

Adolescent↗

PubMed2003

Electrophysiological indicators of phonetic and non-phonetic multisensory interactions during audiovisual speech perception.

We studied the interactions in neural processing of auditory and visual speech by recording event-related brain potentials (ERPs). Unisensory (auditory - A and visual - V) and audiovisual (AV) vowels were presented to 11 subjects. AV vowels were phonetically either congruent (e.g., acoustic /a/ and visual /a/) or incongruent (e.g., acoustic /a/ and visual /y/). ERPs to AV stimuli and the sum of the ERPs to A and V stimuli (A+V) were compared. Similar ERPs to AV and A+V were hypothesized to indicate independent processing of A and V stimuli. Differences on the other hand would suggest AV interactions. Three deflections, the first peaking at about 85 ms after the A stimulus onset, were significantly larger in the ERPs to A+V than in the ERPs to both congruent and incongruent AV stimuli. We suggest that these differences reflect AV interactions in the processing of general, non-phonetic, features shared by the acoustic and visual stimulus (spatial location, coincidence in time). The first difference in the ERPs to incongruent and congruent AV vowels peaked at 155 ms from the A stimuli onset. This and two later differences are suggested to reflect interactions at phonetic level. The early general AV interactions probably reflect modified activity in the sensory-specific cortices, whereas the later phonetic AV interactions are likely generated in the heteromodal cortices. Thus, our results suggest that sensory-specific and heteromodal brain regions participate in AV speech integration at separate latencies and are sensitive to different features of A and V speech stimuli.

Acoustic Stimulation↗

PubMed2003

Cortical operational synchrony during audio-visual speech integration.

Information from different sensory modalities is processed in different cortical regions. However, our daily perception is based on the overall impression resulting from the integration of information from multiple sensory modalities. At present it is not known how the human brain integrates information from different modalities into a unified percept. Using a robust phenomenon known as the McGurk effect it was shown in the present study that audio-visual synthesis takes place within a distributed and dynamic cortical networks with emergent properties. Various cortical sites within these networks interact with each other by means of so-called operational synchrony (Kaplan, Fingelkurts, Fingelkurts, & Darkhovsky, 1997). The temporal synchronization of cortical operations processing unimodal stimuli at different cortical sites reveals the importance of the temporal features of auditory and visual stimuli for audio-visual speech integration.

Cerebral Cortex↗

PubMed2003

Structural (operational) synchrony of EEG alpha activity during an auditory memory task.

Memory paradigms are often used in psycho-physiological experiments in order to understand the neural basis underlying cognitive processes. One of the fundamental problems encountered in memory research is how specific and complementary cortical structures interact with each other during episodic encoding and retrieval. A key aspect of the research described below was estimating the coupling of rapid transition processes (in terms of EEG description) which occur in separate cortical areas rather than estimating the routine phase-frequency synchrony in terms of correlation and coherency. It is assumed that these rapid transition processes in the EEG amplitude correspond to the "switching on/off" of brain elemental operations. By making a quantitative estimate of the EEG structural synchrony of alpha-band power between different EEG channels, it was shown that short-term memory has the emergent property of a multiregional neuronal network, and is not the product of strictly hierarchical processing based on convergence through association regions. Moreover, it was demonstrated that the dynamic temporal structure of alpha activity is strongly correlated to the dynamic structure of working memory.

Acoustic Stimulation↗

PubMed2003

Visual awareness of low-contrast stimuli is reflected in event-related brain potentials.

Visual awareness was studied in 11 subjects by using coherent and scrambled objects as stimuli. The stimuli were presented near the subjective perceptual threshold. Explicitly recognized stimuli elicited a specific negative ERP deflection peaking at 460 ms. Our reaction time experiment suggests that this visual awareness negativity (VAN) is similar to that found previously at a shorter latency. We propose that VAN might reflect the exact moment of becoming aware of the task-relevant stimulus properties and provide an online marker of the temporal dynamics of visual awareness.

Adult↗

PubMed2002

Abstract phoneme representations in the left temporal cortex: magnetic mismatch negativity study.

We investigated the brain mechanisms enabling one automatically discriminate phoneme category irrespective of the large inter-speaker variability in the acoustic features of the voices. For this purpose, subjects were presented with 450 different speech stimuli, each uttered by a different speaker, belonging to three vowel categories, while a 306-channel magnetoencephalogram (MEG) was obtained to record the magnetic counterpart of the mismatch negativity (MMNm), elicited only when sensory memory traces for repetitive sounds are formed in the auditory cortex. Despite this wide acoustic variation, category changes elicited prominent MMNm responses, which were considerably stronger in the left than in the right hemisphere in the right-handed subjects. These results implicate the presence of long-term memory traces for vowels, which can recognize the vowel-specific invariant code enabling correct vowel percept even in the presence of realistic acoustic variation.

Adult↗

PubMed2002

Motor cortex involvement during verbal versus non-verbal lip and tongue movements.

We evaluated left and right motor cortex involvement during verbal and non-verbal lip and tongue movements in seven healthy subjects using whole-head magnetoencephalography. The movements were paced by tone pips. The non-verbal tasks included a kissing movement and touching the teeth with the tongue. The verbal tasks comprised silent articulation of the Finnish vowel /o/, which requires mouth movement similar to that in the kissing task, pronouncing the same self-selected word repeatedly, and producing a new word for every tone pip. Motor cortex involvement was quantified by task-related suppression and subsequent rebound of the 20-Hz activity. The modulation concentrated to two sites along the central sulcus, identified as the motor face and hand representations. The 20-Hz suppression in the face area was relatively similar during all tasks. The post-movement rebound, however, was significantly left-lateralized during word production. In the non-verbal tasks, hand areas showed pronounced suppression of 20-Hz activity that was significantly diminished for the verbal tasks. The latencies of the 20-Hz suppression in the left and right face representations were correlated across subjects during verbal mouth movements. Increasing linguistic content of lip and tongue movements was thus manifested in spatially more focal motor cortex involvement, left-hemisphere lateralization of face area activation, and correlated timing across hemispheres.

Adult↗

PubMed2002

Processing of changes in visual speech in the human auditory cortex.

Seeing a talker's articulatory gestures may affect the observer's auditory speech percept. Observing congruent articulatory gestures may enhance the recognition of speech sounds [J. Acoust. Soc. Am. 26 (1954) 212], whereas observing incongruent gestures may change the auditory percept phonetically as occurs in the McGurk effect [Nature 264 (1976) 746]. For example, simultaneous acoustic /ba/ and visual /ga/ are usually heard as /da/. We studied cortical processing of occasional changes in audiovisual and visual speech stimuli with magnetoencephalography. In the audiovisual experiment congruent (acoustic /iti/, visual /iti/) and incongruent (acoustic /ipi/, visual /iti/) audiovisual stimuli, which were both perceived as /iti/, were presented among congruent /ipi/ (acoustic /ipi/, visual /ipi/) stimuli. In the visual experiment only the visual components of these stimuli were presented. A visual change both in audiovisual and visual experiments activated supratemporal auditory cortices bilaterally. The auditory cortex activation to a visual change occurred later in the visual than in the audiovisual experiment, suggesting that interaction between modalities accelerates the detection of visual change in speech.

Acoustic Stimulation↗

PubMed2002

Probability interrelations between pre-/post-stimulus intervals and ERD/ERS during a memory task.

OBJECTIVES: To investigate the functional relationship between oscillatory electroencephalographic (EEG) components (pre-/post-stimulus intervals) with audio-event-related desynchronization/synchronization (ERD/ERS). METHODS: In an experimental study (9 subjects), the probability-classification analysis of single-trial spectral EEG changes was utilized. Results were compared with auditory ERD/ERS. RESULTS: It was shown that (1) variability of EEG spectral patterns was considerable, (2) EEG activity was different at various task stages, (3) probability measures were different from the results of conventional frequency analysis, (4) probability of trials with alpha- and theta-patterns was characteristically different in various task stages, (5) the occurrence of alpha- and theta-trials were most probable, but not frequent enough to characterize all the trials. The results suggest that the ERD/ERS responses are influenced by EEG characteristics in the pre-stimulus interval, which also have a strong influence on the EEG in the post-stimulus interval. CONCLUSIONS: Alpha- and theta-ERD/ERS responses during memory task performance are not typical for all trials. They reflect EEG changes only in approximately 39% for alpha-activity and approximately 43% for theta-oscillations of all trials, what reflects piecewise stationary EEG structure.

Acoustic Stimulation↗