Large-scale microbiome sampling and sequencing has provided a new window into the diversity of the microbial world. This effort highlights the opportunities to understand critical issues related to human and environmental health, and at the same time underlines the need for quantitative methods to characterize the dynamics and function of microbial communities. To achieve this goal, we need theoretical models that incorporate processes in multiple timescales, stochasticity, imperfect sequence data, and non-stationarity. In this project we introduce a phylodynamics framework that leverages the methods of coarse-graining and non-equilibrium dynamics to extract information about microbial dynamics from uncertain and incompletely-sampled phylogenetic trees. We developed a set of nested theoretical models that combine gradual, ongoing diversification with innovation events on much shorter timescales. We think of these events as arising due to a key innovation, opening the door to a radiation that is then saturated.
We describe a bioinformatics pipeline which takes as input a rooted phylogeny, optionally with taxon abundances, and produces as output a set of estimates for the rates and distribution parameters of candidate stochastic processes generating the phylogeny, a likelihood-ratio test statistic to select among models, and an exact goodness of fit test statistic. We applied this methodology to 13,500 samples drawn from the Earth Microbiome Project, and use it to quantify the balance of gradual vs punctuated diversification. We uncover a novel and overwhelmingly supported dynamical pattern in which (1) fast punctuated speciation bursts occur per unit time in proportion above 1:1 to gradual speciation, and (2) the distribution of those radiation sizes appears to cluster in parameter space (quartile coefficient of variation 0.07) around a power-law exponent with median 3.6. This phenomenon happens across the vast majority of microbial habitats sampled, from hot-springs, freshwater and soils to human- and other host-associated habitats, and suggests the presence of universal features in the way microbes underwent successive innovation-driven punctuated evolution.