Notes

Outline

Non-Linear Modelling and Chaotic Neural Networks Evolutionary and Neural Computing Group Cardiff University SBRN 2000

Overview The Freeman model The Gamma Test Non-Linear Modelling Delayed Feedback Control Synchronisation

The Freeman Model Freeman [1991] studied the olfactory bulb of rabbits In the rest state, the dynamics of this neural cluster are chaotic When presented with a familiar scent, the neural system rapidly simplifies its behaviour The dynamics then become more orderly, more nearly periodic than when in the rest state

Questions... How can we construct chaotic neural networks? How can we control such networks so that they stabilise onto an unstable periodic orbit (characteristic of the applied stimulus) when a stimulus is presented? We are looking for biologically plausible mechanisms

"Principal Contributors" Principal Contributors

An introduction to the Gamma Test Assume a relationship of the form where:  f is smooth function (bounded derivatives)  y is a measured variable possibly dependent on measured variables x1,…,xm  r is a random noise component which we may as well assume has mean zero

Question: What is the noise variance Var(r)? The Gamma test estimates this directly from the observed data (despite the fact that the underlying smooth non-linear function is unknown) It runs in O(M log M) time, where M is the number of data points We can deal with vector y at little extra computational cost

The Details

The Algorithm

An Example

1000 sampled data points with noise variance Var(r)=0.01

Probabilistic asymptotic convergence of G to Var(r)

Using The Gamma Test for Non-Linear Modelling Embedding Dimension Irregular Embeddings Modelling a particular chaotic system

Question: What use is the Gamma Test? We can calculate the embedding dimension the number of past values required to calculate the next point We can compute irregular embeddings  the best combination of past values for a given embedding dimension

Choosing an Embedding Dimension Time-series    ...x(t-3), x(t-2), x(t-1), x(t)... Task is to predict x(t) given some number of previous values Take x(t) as output, and x(t-d),...,x(t-1) as inputs, then run the Gamma Test Increase d until the noise estimate reaches a local minimum This value of d is an estimate for the embedding dimension

An Example The Mackey-Glass Series Time-delayed differential equation Dataset created by integrating from t=0 to t=8000 and taking points where t=10,20,30,....,8000

The Mackey-Glass Time Series

Finding the Embedding Dimension

Finding Irregular Embeddings Given a data set with m inputs, we can select which combination of inputs produces the best model even if there is no noise This gives us an irregular embedding Omitting a relevant input produces pseudo-noise

Pseudo-noise of  a Conical function

Gamma Test Analysis Given the conical function, pseudo-noise is apparent if we leave out either x or y from the model of z Var(r) is the estimate for pseudo-noise variance (M=500)

An Example The Mackay-Glass Time Series

Gamma Scatter Plot for Embedding 111100

Model Construction Neural Network (4-8-8-1) using input mask 111100 Trained using the BFGS algorithm on 800 samples to the MSE predicted by the Gamma Test (0.00032) MSE on 100 unseen samples 0.00040

Iterating the Network Model

Phase-Space Comparison

Control via Delayed Feedback

Controlling the Neural Network

Varying the Stimulus

A Generic Model for a Chaotic Neural Network

Synchronisation Method

Results of Synchronization

Conclusions Given a chaotic time series we can use the Gamma Test to determine an appropriate embedding dimension and then a suitable irregular embedding We then train a feedforward network, using the irregular embedding to determine the number of inputs, so that the output gives an accurate one-step prediction By iterating the network with the appropriate time delays we can accurately reproduce the original dynamics

The significance of time delayed feedback Finally by adding a time delayed feedback (activated in the presence of a stimulus) we can stabilise the iterative network onto an unstable periodic orbit The particular orbit stabilised depends on the applied stimulus The entire artificial neural system accurately reproduces the phenomenon described by Freeman

Synchronisation Results shown by Skarda and Freeman [Skarda 1987] support the hypothesis that neural dynamics are heavily dependent on chaotic activity Nowadays it is believed that synchronization plays a crucial role in information processing in living organisms and could lead to important applications in speech and image processing [Ogorzallek 1993] We have shown that time delayed feedback also offers a biologically plausible mechanism for neural synchronisation

SBRN2000 Group Picture