Neural computations for visual form processing and form-based cognition

When deciding whether to eat an apple, you might focus on its color, firmness and smell. If you’re about to throw it to a friend, you might note instead, perhaps unconsciously, its size and shape. This simple example illustrates a fundamental principle: we can use the same visual input to guide a variety of decisions and actions. How the brain achieves this flexibility is unknown. We will try to answer this question by studying the part of the cortical visual system responsible for pattern recognition — the ‘ventral pathway’ — which serially elaborates the visual information needed to identify objects, and formats that information so that other parts of the brain can use it to guide decisions and actions. Each member of our team has separately helped to discover how different parts of the ventral pathway contribute to vision. Now we will team up to ask in broader terms how it works from end to end, and how it contributes to the performance of complex visual tasks. Working in monkeys — whose visual systems are very like our own — we will record the activity of neurons from many areas in the ventral pathway. In one family of experiments, we will use a consistent and comprehensive approach to study how the visual areas of the ventral pathway create visual representations that allow us to recognize and categorize the contents of our visual world. In a second family of experiments, we will ask how the information in these representations is used to support flexible behavior. For example, at one moment monkeys might be instructed to choose between different objects, say an apple and a banana. But in the next moment, the monkey might choose between different kinds of objects, say fruits (apple or banana) and vegetables. Thus the same visual stimuli can guide two different decisions depending on the situation, and our recordings will reveal the neural computations that underlie this capacity. While our ambitious experimental, model-building and theoretical endeavor is beyond the capacity of any one of us, we will be able to achieve it through the combined, collaborative effort of our team.