Global Sensitivity Analysis for Impact Assessments

Background/Question/Methods

Results of stochastic population models are often sensitive to model structure and input parameter values, which in turn depend on our knowledge of the processes governing population growth.  Sensitivity analysis is used to understand the effect of parameter and model structure uncertainty on model outcomes.  A common sensitivity analysis method is to vary model parameters one-at-at-time and observe the effect of each parameter on model outcomes separately. This type of analysis fails to account for interactions among model parameters. Several recent sensitivity analyses overcome this limitation by varying several, or all, model inputs simultaneously, and using regression techniques to measure the importance of variation in each model input on model outcome. These methods often require carrying out a large number of simulations. We investigated the use of Latin Hypercube sampling as a way to more efficiently sample model parameter space in a global sensitivity analysis, applying our method to a demographic model of an endangered shore bird whose habitat is threatened by sea-level rise. We also performed a global sensitivity analysis on paired simulations, where each model was parameterized with identical randomly generated input parameters except for those parameters associated with the effects of sea-level rise (e.g. population carrying capacity). This analysis allowed us to calculate a relative change in model outcomes (e.g. change in risk of extinction) caused by sea-level rise.

Results/Conclusions

We find no compelling evidence that Latin Hypercube sampling yields any benefit over uniform random sampling of parameter space. We varied eight input parameters related to vital-rates (e.g. adult survival and fecundity) and population characteristics (e.g. carrying capacity), and found that a relatively small number of replicate models (approximately 100) result in consistent measures of variable importance in non-paired simulations.  However, using our paired simulations, many more replicate models (approximately 500) are required to yield consistent measures of variable importance on relative change measures of model outcomes. In many applications such as the one we present here, the goal of a demographic modeling is not necessarily to determine a value for absolute extinction risk, but to assess how different scenarios (management scenarios, estimated changes to ecological conditions, etc.) will lead to changes in extinction risk. Our results suggest that paired simulations with identical parameters except for the fixed effect being assessed allows quantitative assessment of the sensitivity of relative changes in model outcomes to variation of model inputs as well as absolute model outcomes.