Investigating symbol-like reasoning in the brain: Neural mechanisms of compositional action planning

Humans and other animals can rapidly solve new problems — such as imitating a dance from a single observation — by adaptively combining components from prior knowledge to generate new thoughts or action plans. This ability, compositionality, is central to intelligence. Yet its neural mechanisms are unknown. Cognitive models suggest an important role for symbolic knowledge in compositionality: abstract structures and algorithms to support extrapolative reasoning. Despite the success of these models, we do not know whether or how symbolic computations are implemented in the brain. Bridging this gap between symbols and neurons would be a critical advance toward explaining mechanisms of compositionality. To this end, we have developed a new animal behavioral paradigm for compositional reasoning, and we will uncover its underlying neural mechanisms by combining neurophysiology with cognitive and neural network modeling.

In a new drawing task, each ‘problem’ is an image the subject needs to draw. Subjects draw by reusing components of a learned action grammar. An action grammar is a symbolic generative model for drawing, consisting of abstract action primitives and abstract sequencing rules. After training, subjects are tested on their ability to generalize by extrapolating these learned components to use on novel, harder problems. Preliminary behavioral data and computational modeling indicate that subjects solve test problems using past learning of action grammars for one-shot generalization.

Our goal for this pilot project is to determine the neural codes and dynamics underlying this compositional prior knowledge during drawing. We will perform large-scale neural recordings across multiple frontal, motor and parietal cortical areas towards two specific aims.

Aim 1: Identify compositional structure of neural population codes. We will test the hypothesis that neural activity harbors an explicit code for action grammar components. Each component — action primitive or sequencing rule — would be represented in a manner largely independent of other active components. This hypothesis will be compared to an alternative hypothesis based on predicting motor kinematics directly from neural activity, without positing abstract action grammar components as an intermediary.

Aim 2: Identify theoretical principles of compositionality in neural computation. We will develop and test theoretical principles linking neural processing to compositionality via analysis of artificial neural networks (ANNs). This approach leverages the potential of ANNs, optimized to capture neural and behavioral data, to ‘discover’ mechanistic strategies. These strategies make predictions that are testable in neural data. We will build a library of ANNs and identify mechanistic principles in the models that best capture behavioral and neural data. These principles are expected to suggest how neural dynamics within and across areas interact in a compositional manner, including the potential relationship between these dynamics and symbolic computations.

At the end of the two-year funding period, we expect to have taken a critical step toward explaining how symbol-like cognitive components for drawing relate to neural computations in multi-area brain networks.