Information-Theoretic Sequential Framework to Elicit Dynamic High-Order Interactions in High-Dimensional Network Processes

Abstract

Complex networks of stochastic processes are crucial for modeling the dynamics of interacting systems, particularly those involving high-order interactions (HOIs) among three or more components. Traditional measures—such as mutual information (MI), interaction information (II), the redundancy-synergy index (RSI), and O-information (OI)—are typically limited to static analyses not accounting for temporal correlations and become computationally unfeasible in large networks due to the exponential growth of the number of interactions to be analyzed. To address these challenges, first a framework is introduced to extend these information-theoretic measures to dynamic processes. This includes the II rate (IIR), RSI rate (RSIR), and the OI rate gradient (ΔOIR), enabling the dynamic analysis of HOIs. Moreover, a stepwise strategy identifying groups of nodes (multiplets) that maximize either redundant or synergistic HOIs is devised, offering deeper insights into complex interdependencies. The framework is validated through simulations of networks composed of cascade, common drive, and common target mechanisms, modelled using vector autoregressive (VAR) processes. The feasibility of the proposed approach is demonstrated through its application in climatology, specifically by analyzing the relationships between climate variables that govern El Niño and the Southern Oscillation (ENSO) using historical climate data.