Assess Quality of the NOAA NextGen Weather
4-D Data Cube Verification Service
How will the users of the NOAA NextGen Weather 4-D Data Cube understand the quality and uncertainty of the weather products and data provided?
By using widely recognized weather forecast verification techniques, the NextGen Program can provide information about the forecast quality of the data in the NOAA NextGen Weather 4-D Data Cube (4-D Data Cube). Web based services providing verification scores that convey the accuracy of the forecast will be a part of the new capabilities now under development for NextGen.
Weather forecast verification is an essential part of the National Weather Service (NWS) forecast process, to enable forecasters to understand the skill in the routine and warning forecast products, and to allow them to continually improve. Verification also allows the users of aviation forecasts to help quantify the air traffic management decisions made on a daily basis which promote a safe and efficient National Airspace. Today limited capabilities for weather forecast verification of Aviation Services products are provided; however the NextGen Program will enable a significant extension of these critically needed capabilities.
An understanding of a forecast's performance is first necessary before effectively using that forecast for aviation operations and planning. The Network-Enabled Verification Service (NEVS), being developed by the Forecast Verification Section of NOAA's Earth System Research Laboratory, will provide performance information in real time within the NextGen infrastructure as input to the aviation decision-making process and as feedback to the forecast process.
NEVS will ingest forecast and observation data (as shown in Fig. 1) as they become available to provide the latest performance results as soon as possible. To determine forecast performance, NEVS will have multiple ways of associating forecasts to observations, also known as the verification mechanics. One way that the forecast performance will be evaluated will be based upon the context in which the forecast is used for aviation management planning; with criteria specific to that particular planning scenario serving as input to the performance metric. Facets of the planning scenario that will formulate the performance criteria are the forecast's use for strategic or tactical planning, geographical region the plan covers, time of day the plan is created, and time of day for which the plan is valid. Output from NEVS will include traditional verification information, as well as new quality assessment techniques for conveying forecast quality in the context of its use for NextGen.
Figure 1. Inputs and Outputs for NEVS.
Weather forecast verification for NWS products is completed after the forecast products are issued and disseminated to users. For both routine and warning products issued from operational forecasters at national centers and weather forecast offices, verification techniques comparing observed conditions to the forecast conditions are applied to obtain the verification results.
For example, to verify a convective forecast, data for the actual observed convection is used to compare to the issued forecast. In the depiction below (Fig. 2) the verification process for a routine Aviation Services product, the Collaborative Convective Forecast Product (CCFP), is shown. The CCFP is a seasonal product which is generated by human forecasters at the NWS Aviation Weather Center (AWC). A gridded observational data product, the National Convective Weather Depiction (NCWD), is used for comparison to the forecast. Verification results are now routinely computed for the CCFP forecast product over a specified space and time period. When NEVS is operationally implemented as an NWS system, these verification results will be made available to the users of the 4-D Data Cube in real time.
Figure 2. Example of weather forecast verification process. Human generated forecasts such as the Convective Collaborative Forecast Product (CCFP) are compared to observational data such as the National Convective Weather Depiction (NCWD) analysis, and verification results are computed for specified areas over time.
The NWS is working towards implementation of NEVS to support NextGen requirements and to replace the existing legacy verification system used by the forecasters and managers of the NWS AWC, the Real-Time Verification System (RTVS). Weather forecast verification results from NEVS will provide information on the quality of the human generated forecasts and other weather data provided by the NWS in the 4-D Data Cube. NEVS will also maintain an on-going historical performance record of NWS Aviation Services weather products.
Verification will be a critical component of NextGen to inform users about the quality of products and weather information provided to the Federal Aviation Administration (FAA) in the 4-D Data Cube. To meet the need for a common representation of weather information for use at the FAA, capabilities for a Single Authoritative Source (SAS), a subset of the data provided in the 4-D Data Cube, are planned. NEVS will provide performance feedback to users of the products and data provided within this SAS.
To enhance the integration of weather information into Aviation Operations for Air Traffic Management (ATM), the FAA plans to translate weather data provided in the SAS into aviation impacts for key user thresholds. Automated performance information will be needed as input to calibrate weather information during the translation process.
Figure 3. Vision for NEVS in NextGen.
As illustrated in Fig. 3, NEVS will have quality information both for the weather forecasts as well as their translation output, which will facilitate an understanding of the relationship between the quality of weather and the quality of its translation for aviation planning. This can be used to provide feedback into the raw forecast performance information that is provided to the SAS, enabling optimization of the use of the forecast toward this particular Weather Translation algorithm.
We envision that the quality information obtained from NEVS will be applied as feedback to continually improve the NWS forecasts and the application of these forecasts in FAA operations, all contributing to enhancing the services provided, to enable the NWS and the FAA to meet NextGen objectives for safer and more reliable air transportation.
For additional information on NEVS and to view demonstrations of selected verification capabilities provided by NEVS see: http://esrl.noaa.gov/nevs/
By 2013 when implemented at the NWS for NextGen, NEVS will provide:
- Capabilities for verification of NWS Aviation Services products.
- Capabilities for verification of data in the 4-D Data Cube.
- Net-enabled verification services to convey forecast accuracy.
- Verification reporting for the FAA's Quality Management System (QMS).
- Objective information for NextGen SAS contents decision-making.