Title: Multimodally encoded spatial images in sighted and blind

NIH grant R01-EY016817-01

J.M. Loomis, UCSB (PI), N.A. Giudice (UMaine-PI), and R.L. Klatzky, CMU

Brief Abstract

Humans can perceive surrounding space through vision, hearing, touch and sensed consequences of movement.  As they perceive and interact with their environment, people establish, from perceptual processing, an ongoing representation of the physical layout of objects and locations.  This representation remains at a perceptual level as long as it is supported by sensory stimulation.  Perhaps surprisingly, when such stimulation ceases, as when the eyes close or a sound source is turned off, the perceptual representation also ceases; yet, people remain able to direct actions appropriately in space.  This proposal investigates the representation of spatial layout that remains available in the absence of direct perceptual support, and hence serves to guide action.  We call it the "spatial image".  We propose that spatial images are (1) fully three-dimensional and externalized, (2) capable of being formed in working memory, both from perception by way of multiple sensory modalities and from constructive spatial processes (imagination or retrieval from long-term memory), and (3) amodal in nature (not depending on the input source).

Our proposed research will further knowledge about spatial images produced by visual, haptic, and auditory input. Our research consists of theoretically-based experiments involving sighted and blind subjects. All of the experiments rely on logic to make inferences about internal processes and representations from observed behavior, such as verbal report, joystick manipulation, and more complex spatial actions, like reaching, pointing, and walking. Our experiments are grouped into 3 topics. The first topic deals with the development of spatial images through touch, both direct and when mediated by holding a tool, and whether spatial images can be mentally re-scaled at will. The second topic is the testing of the amodality hypothesis: that regardless of the sensory source, spatial images formed from different modalities function in the same way. The third topic is concerned with whether spatial images are equally precise in all directions around the head, in contrast to visual images which are thought to be of high precision only when located in front of the head.