Strategy Change

This project arises from our deep understanding of the rapid biological and cognitive processes displayed by strategy changes in coping with changing environments. This research on decision making in human and animal brains provides a platform for developing robust decision support systems that operate in dynamically changing scenarios in the style of brains.

The goal of this project is to investigate the mechanism of strategy change in biological and technical systems under controlled conditions. Based on data of animal behavior experiments, strategy changes seem to be a rather fast transition in time from one behavior to another. This phenomenon can be described as the 'aha' effect. Linear learning theories like classical conditioning and reinforcement learning fail to explain this sudden qualitative change in the behavior. The qualitative change of behavior (strategy change) seem to occur not only after the environment changed and the learning system is forced to adapt to a new strategy, but such strategy changes can be observed during stable environmental conditions as well; i.e., stable in terms of stable contingency conditions.

This project aims at improved understanding of the nature and functional role of abrupt, large-scale state transitions in complex neuronal systems as the basis of cognitive strategy change.

We exploit our experimental and theoretical understanding of a particular rodent learning model to simulate the neuronal mechanisms of instantaneous strategy change.

We develop an algorithmic formulation of the neurocomputational principles, and apply it in the engineering example of autonomous vehicle control.

Detailed analysis of the mechanisms underlying rapid strategy change in brains will allow both this research team and other groups to equip various man-made systems with the fundamental property of insightful cognition. This work addresses important societal needs by creating the foundations of cognitive engineering systems supporting emergency response to natural disasters and cyber security threats by adversaries, as well as optimized control of autonomous vehicles under complex operating conditions.

US and German scientists work on this project. American scientists based at the Dept. of Mathematical Sciences, University of Memphis, TN, develop mathematical models of strategy change, to interpret neuroscience observations. The German team conducts experimental work, coordinated by the Leibniz Institute of Neuroscience, Magdeburg, Germany. The joint work program serves the following goals: a) develop testable predictions from the theoretical hypotheses of strategy change; b) measure neural patterns in animals and perform these tests on the experimental data; and c) use the results to refine the model and thereby suggest concepts for improved performance of technical systems. The results are of benefit for better understanding of biological and artificial cognitive systems.

The German partner conducts experiments with Mongolian gerbils and it provides the obtained data to the American team for further analysis and interpretation. New and modified experiments are designed and executed based on the feedback from the US team. The gerbils are chronically implanted with multi-site electrodes in three different brain structures, namely, auditory cortex, striatum, and prefrontal cortex. After a short recovery period trained in either a Go/NoGo avoidance detection or a Go/NoGo avoidance discrimination task. During behavior neural signals, such as local field potentials and action potentials are recorded. In both tasks a gerbil is placed in a 2-compartment cage (shuttle box) with a little hurdle separating the two compartments. The animal is trained to cross this hurdle in response to an acoustic signal, called the condoning stimulus (CS)) in order to avoid an aversive electrical stimulation ("foot shock") via the floor grid. This is achieved by consequently preceding the foot shock with the sensory stimulus.

The US team develops models to explain experimental data from German team regarding sudden strategy changes in stable conditions in the framework of neurodynamics. The overall research strategy is illustrated in Fig. 1. The goal is to describe the learning system (the brain) by hierarchical coupled oscillators producing complex spatial-temporal patterns of simulated neural activity. These patterns can be explained as trajectories in phase spaces using tools of nonlinear systems theory. In our theoretical approach, the learning system continuously accumulates incremental changes during learning, which can be described by established learning rules and models. However, above a certain threshold, the system changes its properties in a qualitative manner. The overall goal of our team is to interpret such qualitative changes in the context of identified strategy changes in the gerbil.