Statistics of Attractor Embeddings in Reservoir Computing
A recent branch of AI or Neural Networks that can handle time-varying signals often in real time has emerged as a new direction for signal analysis. These dynamical systems are usually referred to as reservoir computers. A central question in the operation of these systems is why a reservoir computer (RC), driven by only one time series from a driving or source system of many time-dependent components, can be trained to recreate all dynamical time series signals from the drive leads to the idea that the RC must be internally recreating the drive dynamics or attractor. This has led to the possibility that the RC is creating an embedding of the drive attractor in the RC dynamics. There have been some mathematical advances that move that argument closer to a general theorem. However, for RCs constructed from actual physical systems like interacting lasers or analog circuits, the RC dynamics may not be known well or known at all. And many of the existing embedding theorems have restrictive assumptions on the dynamics. We first show that the best way to analyze RC behavior is to first treat it properly like a dynamical system, which it is. This will lead to some conflict with existing ideas about RCs, but also a clarification of those ideas. Secondly, in the absence of complete theories on RCs and attractor embeddings, we show several ways to analyze the RC behavior to help understand what underlying processes are in place, especially regarding if there are good embeddings of the drive system in the RC. We show that a statistic we developed for other uses can help test for homeomorphisms between a drive system and the RC by using the time series from both systems. This statistic is called the continuity statistic and it is modeled on the mathematical definition of a continuous function. We show the interplay of dynamical quantities (e.g. Lyapunov exponents, Kaplan-Yorke dimensions, generalized synchronization, etc.) and embeddings as exposed by the continuity statistic and other statistics based on ideas from nonlinear dynamical systems theory. These viewpoints and results lead to a clarification of various currently vague concepts about RCs, such as fading memory, stability, and types of dynamics that are useful.