Ecologists often face a quandary when choosing how to analyze their data. Should I transform data and use a traditional analysis, or use a modern computationally intensive analysis? The traditional analysis is clearly an approximation; what is not commonly understood is that many modern analyses are also approximate. Hence, the choice of analysis is not clear. I consider various ways to analyze weed counts in a large multi-location evaluation of weed management strategies. Farmer’s fields, the main plots, were classified into one of seven crop-rotation treatments, then divided in half. One of two weed management strategies was randomly assigned to the half field. We consider the analysis of the number of weeds present at a specific point in time. The weed counts are overdispersed relative to a Poisson distribution.
The traditional analysis is to log transform the counts then assume a normal distributions for the random effects. This leads to the standard analysis of a split-plot study. A modern alternative is use a generalized linear mixed model assuming an overdispersed Poisson distribution and an additional random component to account for field-field variation. The GLMM can be specified in different ways.
Results/Conclusions
We find that the split-plot analysis of transformed data has empirical rejection rates are close to the nominal 5% rate for all tests. The most common analysis of overdispersed Poisson gives conservative tests of the main plot effects (empirical rejection rate < 5%) and very liberal tests of the split plot main and interaction effects. When the overdispersion is moderate (variance associated with overdispersion > 1), the empirical rejection rate can exceed 60%. for a nominal 5% test. An alternate specification of the model has an acceptable type I error rate.
Modern analyses are not always better than traditional analyses. For overdispered split-plot count data, traditional analyses are similar to some modern analyses. Other modern analyses are badly behaved and should not be used.
