Predicting Tipping Points in Complex Systems
The world is full of sudden changes that can be hard to forecast ahead of time. For example, ecologists have revealed that regional mass extinctions of species can occur suddenly as the environment changes gradually (not suddenly). In the past two decades, analysis of data from ecology, epidemiology, and other fields has identified several statistical markers that can anticipate such abrupt changes before they occur (Fig. 1). Although such “early warning signals” are qualitatively successful, they are rarely able to predict exactly when a sudden change will occur. Zijia Liu of Tongji University in China and his colleagues devised an innovative machine-learning method to do just that [1]. Given relatively short-timescale observations of a complex system—for example, the numbers of individuals of different species in an ecosystem—their predictive framework quantitatively anticipates when a sudden change will occur, across different types of dynamics, networks, and scenarios.
Ecosystems, power grids, and living organisms are all complex systems that can experience regime shifts, where a global parameter (such as species populations or electricity output) abruptly changes value. These shifts, also called tipping points or critical transitions, can occur even when their surrounding environment is relatively stable. Research into these regime shifts use tools from statistical physics, dynamical systems theory, and other areas to analyze observational data and anticipate sudden regime shifts before they occur. Earlier work led to a list of data markers that can warn that a system is about to shift [2]. The sample variance and lag-1 autocorrelation (the correlation between data points taken one time period apart) of the observed time series are among the most commonly used early warning signals. In generic complex systems, they increase as the system approaches the impending regime shift.
However, these signals do not necessarily work well for networked systems—ones that consist of seemingly independent but interacting entities [3–5]. Furthermore, finding an early warning signal that alerts to an impending regime shift is not as difficult a problem as finding one that quantitatively predicts when a regime shift will occur. Recent work in ecology has tackled the latter problem by developing a predictive model [6], but the method has yet to be generalized. A separate trend in early-warning-signal research is to use machine learning, which has become more common in physics research [7]. For example, a machine-learning algorithm was able to identify the type of regime shift and to provide early warning signals when supplied data from a host of different complex systems [8]. However, previous machine-learning methods have not yet addressed quantitative prediction of regime shifts for various types of dynamics on networks, which is what Liu’s team has now achieved [1].
By assessing the performance of various machine-learning models, the researchers decided to use a neural network architecture that is composed of layers of so-called graph isomorphism networks (GINs), followed by layers of so-called gated recurrent unit (GRU) neural networks. The GIN layers take as input time-series data observed at various nodes of the network—for example, each node might be a geographic location and the data might track the number of organisms or the amount of precipitation at that location over time. The GRU neural network layers receive the output of the GIN layers and detect recurring patterns in the time-series data; recurrent neural networks are generally suited for time-series data. In this manner, the GIN–GRU neural network predicts when the networked system will undergo a regime shift.
Liu and his colleagues validated their GIN–GRU predictive method on numerical simulations of dynamical systems, such as synchronization transitions in coupled oscillators, and on real data from observations, such as those of vegetation changes in Central Africa as mean annual rainfall levels have gradually decreased. They also performed robustness tests for the predictor and, in addition, showed its transfer ability—meaning that it can use knowledge gained from a previous task to improve performance on a related one. Such transfer-learning ability is important because when we want to predict the tipping point in a novel situation, long-term observations may not be available. In this situation, pretraining the neural network predictor with different but related—and sufficiently abundant—data would get the algorithm ready so that it can reasonably succeed with a relatively small amount of data from the target system.
What comes next? The researchers discussed their goal of decreasing the required data length, which they currently set to 20 time points per node. This is an important research direction because only a few data points per node may be available in a given environment before it gradually shifts toward a different state. Furthermore, some nodes may be more useful than others for constructing early warning signals. Future work will also include improving transfer learning across different networks, such as ones that have different numbers of nodes and different dynamics. How their predictor performs on real data and contributes to applications beyond physics, by collaborating with experts in ecology or psychiatry, for example, will also be exciting.
References
- Z. Liu et al., “Early predictor for the onset of critical transitions in networked dynamical systems,” Phys. Rev. X 14, 031009 (2024).
- M. Scheffer et al., “Anticipating critical transitions,” Science 338, 344 (2012).
- A. C. Patterson et al., “When and where we can expect to see early warning signals in multispecies systems approaching tipping points: Insights from theory,” Am. Nat. 198, E12 (2021).
- N. G. MacLaren et al., “Early warnings for multi-stage transitions in dynamics on networks,” J. R. Soc., Interface 20, 20220743 (2023).
- N. Masuda et al., “Anticipating regime shifts by mixing early warning signals from different nodes,” Nat. Commun. 15, 1086 (2024).
- H. Zhang et al., “Estimating comparable distances to tipping points across mutualistic systems by scaled recovery rates,” Nat. Ecol. Evol. 6, 1524 (2022).
- G. Carleo et al., “Machine learning and the physical sciences,” Rev. Mod. Phys. 91, 045002 (2019).
- T. M. Bury et al., “Deep learning for early warning signals of tipping points,” Proc. Natl. Acad. Sci. U.S.A. 118, e2106140118 (2021).