Information is a key concept for understanding complex biological, economic and social systems. However, there is still no solid framework for analyzing non-intuitive high-order phenomena that take place in complex scenarios involving three or more agents. Recent efforts for addressing this issue by providing a multivariate extension of Shannon’s information theory often rely in one of two alternative notions: the total correlation (TC) [1] or the dual total correlation (DTC) [2]. These two quantities have been explored by disjoint communities —the former being preferred in computational neuroscience and information geometry, and the latter being closely related to partial information decomposition (PID). Although both the TC and DTC provide consistent non-negative extensions of mutual information, their relationship is unclear and their numerical difference is still treated as “enigmatic information” [3]. In this work we show that only DTC measures actual shared information (in the Shannon sense), while TC quantifies the strength of the system’s constraints. For this purpose, we introduce a novel method for providing a hierarchical decomposition of both TC and DTC that can be represented in a double-diamond diagram, where each path linking the top and bottom nodes corresponds to a different decomposition. Furthermore, we show that these paths provide diverse decompositions for the enigmatic information, which can now be understood as constraints minus shared information. We illustrate with examples how the double-diamond diagram structure, combined with the decompositions of the enigmatic information, can provide new insights about information dynamics in complex systems. We also explore the limitations of this approach and suggest how it can be integrated with the PID framework, providing a complementary coarse-grained description of the structure of interdependencies of many variables.
