COS 182-2 - Customized web-based services to access the Daymet product: Analysis of user-based downloads provide insights into how scientists access large, complex data for their research needs

Friday, August 11, 2017: 8:20 AM
C122, Oregon Convention Center
Michele M. Thornton, Yaxing Wei, Ranjeet Devarakonda, Alison G. Boyer, Suresh K.S. Vannan and Peter E. Thornton, Environmental Sciences Division & Climate Change Science Institute, Oak Ridge National Laboratory, Oak Ridge, TN
Background/Question/Methods

The ORNL DAAC (a NASA EOSDIS data center) archives and distributes Daymet. Version 3 of this data product consists of gridded surface weather variables available on a daily time-step for 1980 - 2016 at a 1-km spatial resolution. The spatial extent of Daymet V3 is continental North America, Hawaii, and Puerto Rico. Though this entire data product is freely available for download as CF-compliant netCDF files, many scientists and researchers outside the climate and modeling community find these file sizes and formats unfamiliar and challenging. NASA DAAC’s provide reliable, robust services to users whose needs may cross the traditional boundaries of a science discipline. As a commitment to different scientific user communities and in an effort to maximize the usability of Daymet data, the ORNL DAAC has developed specialized tools and services that allow this multi-variable data product to be downloaded in user-specified subsets (both spatial and temporal) and formats. Services based on OPeNDAP architecture, THREDDS Data Service, and REST APIs provide interactive Web-based GUIs that support data visualization and download and also allow download through URLs with defined spatial and temporal parameters. These REST API’s also enable download automation and compatible client software, e.g. R, customization.

Results/Conclusions

The tools and web-based services provided by the ORNL DAAC greatly increase the use and usability of this data set. As evidenced by the number and volume of data downloads, there is a real and documented need for daily, high resolution, gridded surfaces of meteorological data in standardized, interoperable formats that are both readily available and ingestible into common analysis packages and modeling frameworks. Since 2012, over 80 published journal articles cite the use of the Daymet data product in research products covering fields such as wildlife biology, terrestrial vegetation growth, hydrology, insect/pest management, and climate characteristics. Several governmental organizations have downloaded Daymet data resulting in a total number of orders exceeding well over 200,000 requests since 2012. This is a data volume of more than 261 GB. These include DOE, DOI, EPA, NASA, NOAA, USDA and state agencies. On a weekly bases, a service that provides automation of asci file downloads of single point locations often exceeds over 100,000 queries. A netCDF Subset Service has been utilized with over 200,000 downloads since 2016. These results demonstrate the need for data centers to continue to develop specialized data access services for data that have high value to the scientific community.