Estimating the mutual information rate of short time series from coupled dynamic systems

Abstract

Mutual Information Rate (MIR) quantifies the dynamic coupling between two stochastic processes by measuring the information exchanged per unit time. It is particularly valuable for analyzing complex biological systems, as it captures both linear and non-linear interactions, providing a comprehensive view of system dynamics. Since MIR is theoretically defined for infinite-dimensional stochastic processes, estimating it in practical situations-where only short and stationary realizations are typically available-remains a significant challenge. This work introduces a model-free method based on k-nearest neighbor (kNN) search and compares it with a well-established parametric approach based on linear Vector Autoregressive (VAR) modeling, assuming Gaussianity of the processes. Both estimation strategies compute MIR by decomposing it into components including individual entropy rates, joint entropy rates, transfer entropies, and instantaneous shared information, which relate to broader concepts of complexity and information transfer. The two techniques are evaluated through simulations of coupled linear and non-linear systems (unidirectional and bidirectional), then applied to two real-world systems: heart rate (HR) and respiration (RESP) time series from a patient with sleep apnea, and an ecological predator-prey interaction between Didinium nasutum and Paramecium aurelia. Simulations show that the kNN estimator is biased depending on embedding dimension, while the linear estimator struggles with highly non-linear systems. In real data, both approaches reveal weak coupling between HR and RESP during apneic episodes, whereas the predator-prey system exhibits bidirectional interactions. The estimation of MIR and its components from short datasets is feasible, and the choice between linear and kNN estimators depends on data type and the time series relationship, highlighting the importance of tailored approaches.