Sensory substitution, and the third kind of "qualia"

"Qualia" to some philosophers refers to the absolute, unique and subjective quality of a conscious sensory experience, which may not be "explained away" by any sort of neurophysiological account or model. Whereas we do not endorse to the opinion that the qualia is the "hard" (i.e. impossible in principle) problem for science, we still agree that the current visual perceptual sciences failed to explain the "absolute, unique and subjective quality of conscious sensory experiences." In this context, we may be able to find significant insights in the latest progresses in sensory substitution. The "vOICe" is one of the currently available visual substitution devices based on auditory inputs, primarily for blind people. There are some "super users" who claim "visual" experiences. Moreover, at least one of them showed neural activity in higher-level visual cortical areas in fMRI, when engaged in a shape discrimination task relying on this device. In principle, we may come up with a brief list of psychophysical and neuroscientific criteria to acquire new "visual" experiences, and I will describe some pilot data that we recently obtained along this line, including (1) mapping of egocentric space via the device, (2) accomplishment of some form of perceptual constancy by training, and (3) evidence for intrinsic mapping (synesthetic) between auditory and visual modalities, that enables untrained subjects to discriminate seemingly complicated surfaces. The results suggest that qualia, if still want to use such a word, should be understood with regard to adaptive behavior. They also suggest that what such training/experience accomplishes is not just a visual, nor auditory quality of experience. Rather, it should be characterized best as the third kind of qualia. Both this unique type of qualia and the intrinsic crossmodal mapping provides scientists and artists with ample opportunities for exploration.


Biography
Shinsuke Shimojo is an experimental psychologist/cognitive neuroscientist, with long-standing interests in visual psychophysics and their applications to visual illusions, cross-modal plasticity, human emotion, preferences, and decision-making. He received his B.A. and M.A. degrees from the University of Tokyo (1978, 1980), and his Ph.D. from the Massachusetts Institute of Technology (1985). He is currently Gertrude Baltimore Professor in Experimental Psychology in the division of Biology/Computation and Neural Systems at the California Institute of Technology. His laboratory at Caltech has been applying quantitative psychophysical techniques to understand human implicit perception and social behaviors, with applications of eye tracking, fMRI, EEG, TMS and tDCS, and other techniques. He has approximately 150 publications, including multiple papers in high-impact journals, such as Nature, Science, Nature Neuroscience, and Neuron. He is also known for his collaborations with artists, and a science column (ASAHI WEBRONZA).