In order to monitor the effects of changing climate, altered biogeochemical cycles, and human drivers on ecological systems, increasing numbers of automatic sensors are being deployed to support basic research and natural resources management. Many of these sensors are digital cameras, which capture images at high frequency across broad geographical extents. Examples include camera traps, satellite imagery, environmental webcams, and mobile devices used for citizen science. While the resulting images promise to provide new scientific insight, their sheer volume is a challenge for researchers and managers who need to turn images into analyzable data.
The PhenoCam network consists of more than 200 elevated automatic digital cameras across North America that take hourly pictures of diverse ecosystems, including deciduous forest, evergreen forest, grasslands, shrublands, wetlands, and tundra. One obstacle to automatically processing these images is the occurrence of snow. We used crowdsourcing to label ~200,000 PhenoCam images across 133 cameras. Each image was viewed by three volunteers, who classified it as not having snow, having snow on the trees, or having snow on the ground but not on the trees.
Concerned about the sustainability of crowdsourcing, we then turned to state-of-the-art Deep Learning techniques to classify images as having snow or not. Deep Learning is a branch of machine learning that is currently an intense area of research in computer science. Applications of Deep Learning to computer vision have resulted in computers being able to recognize objects, classify scenes, and recognize faces. We implemented an off-the-shelf Deep Learning platform to classify the occurrence of snow in the PhenoCam images.
Results/Conclusions
Compared with an expert-annotated subset of images, the crowdsourced labels were 99% accurate in determining whether or not an image had snow. The off-the-shelf Deep Learning platform with naive scoring was 92% accurate, and preliminary results indicate that more sophisticated scoring techniques will increase its accuracy.
Crowdsourcing and Deep Learning are generally applicable techniques for the growing problem of ecological image classification for large datasets. We discuss how to implement crowdsourcing and Deep Learning, as well as the tradeoffs of each for turning large volumes of images into scientifically analyzable data.