The history of life has been typified by fluctuations, large and small, in the diversity of extant lineages. It remains contested what statistical or biological mechanisms drive these fluctuations. Candidate hypotheses have ranged from simple random walks, to density dependent diversification, to models of self-organizing complex systems. While macroevolution (i.e. the diversification of lineages) does not follow a random walk at the global scale, we ask, could diversification within lineages conform to such a simple model? Further, could complexity at the global scale be the result of temporal turnover in diverse lineages that vary in the volatility of their individualistic random walks? We expand and adapt super-statistical theory, originally from economics, to explain complex patterns in the diversification of life that have previously been presumed as evidence for underlying non-linearity and self-organizing coevolution. We use least squares, maximum likelihood and simulation methods and data from paleontological data bases to assess the validity of the random walk process operating within clades, but differently across clades.
Results/Conclusions
Our theory accurately reproduces the observed paleontological patterns, and does so with no free parameters. This is strong evidence that no further theoretical complication is needed in understanding fluctuations in diversity through time. Our theory suggests that the volatility of clade-specific random walks, a parameter that is constant within clades, but changes between clades, could be constrained phylogenetically and potentially be under natural selection favoring less volatile clades through time, as has been independently proposed in other studies. We find tentative support for this hypothesis in that volatility tends to decrease through time, but a complete phylogenetic analysis will be necessary. In simulating diversification trajectories, we note that our simplest theory under-predicts the variance of the process. We propose two compatible candidate hypotheses, one that clades experience density-dependent diversification, competing with and dominating other clades at different stages in their diversification; and two that major events of speciation and extinction are correlated across clades as was perhaps the case in the Cambrian explosion and during mass extinctions. This hypothesis comes to bare in understanding the current biodiversity crisis as potentially a "high correlation" event in which extinctions occur across diverse clades.