Sensitivity to sampling bias for two methods of selecting optimal complexity in niche models of a Malagasy rodent

Background/Question/Methods:

Ecological Niche Models (ENMs) are widely used, yet optimal complexity and violation of modeling assumptions remain outstanding issues. We analyzed two strategies for selecting optimal ENMs (AICc and a jackknife approach) while simultaneously lessening sampling bias via spatial filtering. We did so for a species with few records, the endemic Malagasy rodent Eliurus majori (subfamily Nesomyinae), using 19 bioclimatic variables and MaxEnt. We varied model complexity, employing various combinations of feature classes and regularization-multiplier values. For the jackknife approach, we calculated the average omission rate and AUC of the withheld (test) records for each feature-class/regularization-multiplier combination. In contrast, AICc uses internal testing to balance complexity and goodness-of-fit. 

Results/Conclusions:

For each dataset, the models selected as optimal by AICc were less complex models than for the jackknife. With the unfiltered dataset, AICc (but not jackknife) selected a model that had high omission, indicating overfitting to training data. In contrast, using the filtered dataset, both techniques selected models with similar (low) omission rates. However, despite this similarity in performance of the filtered dataset, the respective optimal settings led to predictions that were only 85% similar in geographic space. These results highlight the sensitivity of AICc to violation of the assumption of unbiased sampling, and suggest that the jackknife approach may be more robust in this regard.