OOS 5-10
Reducing uncertainty through data-driven model development
Ecological forecasts have diverse sources of uncertainty, and the value of an ecological forecast is directly related to its accuracy and precision. We apply process control and integration testing to target model development toward more precise ecological forecasts. Benchmarks and skill metrics are used to quantify changes in the uncertainty of our ecological predictions that result from changes in parameterization and model structure. This approach reduces the uncertainty associated with the addition of new features to a model by testing that the software implementation of a model is consistent with our conceptual model. Furthermore, the use of fixed benchmark data and software version control makes it easier to isolate the source of changes in model performance. This form of integration testing facilitates iterative development by providing immediate feedback on the effects of each step in model development.
As an example of this approach, we will review our experience adding the capacity to simulate tree growth to an existing model of C4 grasses (BioCro). In this example, we illustrate the use of uncertainty as an empirical metric to guide efficient and effective research.
Results/Conclusions
The use of model skill metrics during model development allowed us to iteratively evaluate and reassess the focus of our model development efforts. Model skill improved substantially with the introduction of well established models of tree physiology: the Farquhar model of C3 photosynthesis, addition of a woody biomass pool, and deciduous phenology. Subsequent refinements were less effective at improving model skill.
The relationship between effort and model uncertainty provided an empirical foundation on which to iteratively identify and reassess the focus of our model development efforts. The progressive improvments in skill helped us identify how well the incorporation of specific mechanisms improved model skill. At the same time, we could confirm our predictions of which model components are most important, and identify targets for further research.
1990 M Street, NW Suite 700, Washington, DC 20036
phone 202-833-8773
fax 202-833-8775
email esahq@esa.org
. All rights reserved.