SHOUT Data Management and Archive

The NOAA UAS Data Management Plan (DMP) was developed within the framework of the Environmental Data Management Committee (EDMC) for handling UAS instrumentation package data and covers acquisition, ingest, quality control, integration, visualization and archiving. Recommendations from that plan were used to prototype and demonstrate all phases of the UAS data lifecycle from real-time operations to post analysis and research (Figure Below).

New data distribution and web-based visualization tools accessed using the NASA MTS and NOAA GTS, in addition to quick access links, provided the capability for users to dynamically toggle on-and-off the various observation layers, adjust layer transparency, and visualize Global Hawk observations along with various satellite observations. Post flight, data was grouped and made available for further analysis allowing users to access previous observations.

During SHOUT 2016, the project made improvements to existing operational and research data ingest, and archival strategies, in addition to improvements to integration and visualization through the provision of several new real-time products by the instrument teams within MTS, initial efforts to archive of data in the Meteorological Assimilation Data Ingest System (MADIS), and team coordination. MADIS, a NWS operational system, provides near real-time data pathways to NOAA operational and research systems, as well as the greater meteorological community is a potential pathway to operations and archive of Global Hawk observations. Once the Global Hawk data sets are fully realized in MADIS, users may access the data through MADIS interfaces or NOAA's Data Catalog. However, to view text, xml, or csv observations user will need to download the MADIS API.

Real-time data sets were made available on the NASA MTS and SHOUT websites throughout the mission. Meteorological observations were placed into GTS, and the SHOUT NRD-94 'minisonde' in-situ data were assimilated into a variety of both operational and research NWP forecast models (Table Below). Because of the criticality of providing accurate input for such applications, especially into operational NWP models, heavy emphasis was placed on quality assurance and processing of all utilized observations.

Summary of model data use from the SHOUT 2016 campaign

NWP Forecast Model Name Model Type: Operational -vs- Research Domain Type Source Nation / Institution
Hurricane Weather Research and Forecasting (HWRF) Operational Regional U.S. / NOAA Hurricane Research Division (HRD)
Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) Operational Regional U.S. / Navy
Navy Global Environmental Model (NAVGEM) Operational Global U.S. / Navy
European Centre for Medium-Range Weather Forecasts (ECMWF) Operational Global Europe (independent intergovernmental)
United Kingdom Meteorological (UKMET) Model Operational Global United Kingdom
Hurricane Weather Research and Forecasting System - Research version (HWRFx) Research Regional U.S. / NOAA Hurricane Research Division (HRD)
Weather Research and Forecasting Model - Advanced Research version (WRF-ARW) Research Regional U.S. /Pennsylvania State University and the National Center for Atmospheric Research (NCAR)
Fifth-Generation Mesoscale Model (MM5) Research Regional U.S. / Pennsylvania State University and the National Center for Atmospheric Research (NCAR)

Active data on the current project is available from the following three web addresses:

HS3 Project (select Data Products)
NOAA Meteorological Assimilation Data Ingest System (MADIS)
NOAA SHOUT Preliminary Data Archive