OPS 2-6
Implementation of NEON’s quality assurance and quality control analyses and their quantification though a quality metric framework
The National Ecological Observatory Network (NEON), set to begin full operations around 2017, will collect and disseminate an array of ecological data across the United States for a period of 30 years. Accordingly, it is essential that NEON assesses and populates the validity of its data. While manual “eyes on” approaches to data quality assessment have dominated in the past, the magnitude of data generated by NEON requires an automated approach. This has resulted in the automation of a suite of initial quality assurance and quality control (QA/QC) analyses. Yet, the interpretation of the resulting data quality flags can become quite challenging with large data sets. Therefore, NEON has developed an automated framework to summarize data quality information and facilitate interpretation by the user. Briefly, the framework consists of first compiling data quality information and then presenting it through two separate mechanisms; a quality report and a quality summary. The quality report presents the results of specific quality analyses as they relate to individual observations. The quality summary takes a spatial or temporal aggregate of each quality analysis and provides a summary of the results. Additionally, this framework can aid problem tracking and resolution, should sensor or system malfunctions arise.
Results/Conclusions
With initial NEON data beginning to stream in from tower sites in the Ordway Swisher Biological Station, FL and Sterling, CO, NEON has begun to implement this framework for terrestrial sensor data. This has enabled NEON to assess initial data quality and identify problem areas that otherwise would have been difficult to pinpoint due to the quantity of data collected. To put in perspective the amount of collected data quality information and the ability of this framework to condense it, there will be over 150 terrestrial sensor observations made at a typical NEON site. Generally, 1 and 30-minute averages are produced from sensor observations that are typically acquired at a rate of 1 Hz and include 8 different QA/QC analyses. Accordingly, the QA/QC results for terrestrial sensors are upwards of 1*108 a day, 2*109 a month, and 3*1010 a year. However, through the presented framework the QA/QC information can be condensed by roughly 4 orders of magnitude for 1-minute averages and 6 orders of magnitude for 30-minute averages.