First Team Charter
NWS Hydrologic Forecast Verification
June 5, 2007 - updated on May 7, 2009
Vision: River forecast verification tools and information will be readily available to users including forecasters, service hydrologists, managers, and the general public. Verification information will be meaningful to each user group. RFC forecasters will generate and communicate river forecast verification results and identify shortcomings to be addressed through software, system, or information enhancements. Ultimately, forecast verification will be successful when its results determine actions within each user group.
Statement of the Problem: Currently, information on NWS river forecast performance is limited in scope and generally not communicated to most user groups. In recent years, nationally supported verification software has been developed which has great potential to address user needs. However, this software remains largely untapped.
Mission: To communicate meaningful river forecast verification information to user groups including forecast users, forecasters, service hydrologists, and management using existing software (IVP and EVS). This mission includes three major components: (1) developing understanding of verification statistics and concepts, (2) developing expertise with IVP and EVS software, and (3) developing standardized verification strategies to effectively communicate results to identified end users while ensuring verification needs are met.
The team will develop a final report by September 30, 2009 that proposes standardized verification strategies to effectively communicate verification results to identified end users. To accomplish this, each RFC focal point will write a brief report describing a verification case study that identifies a specific user group, includes river forecast verification results, and highlights unmet needs. The team leader will coordinate with the RFC verification focal points to ensure the verification case studies consider the broad spectrum of hydrologic products and users. Standard verification strategies will be identified through the case studies.
The team will normally meet monthly via teleconference. Team members will learn about verification statistics and concepts and existing river forecast verification tools at the RFC verification workshop August 14-16 in Salt Lake City. Following the workshop, each team member will lead a call describing a forecast verification problem and their results using the IVP or EVS software. Through this process, the team will develop expertise with verification methodology and with the software. During and following this series of teleconferences, the team will develop standardized verification strategies for specific user groups. The team should realize the success criteria defined here no later than September 30, 2009. Also the team will continue to hold meetings in the future to further discuss the proposed verification standards and share progress on verification science, software and case studies made in the NWS, other agencies, and academia.
No travel beyond the RFC Verification workshop is authorized for these goals.
Proposed Team Membership: Team will be comprised of 13 RFC verification focal points and a sub-set of former members of the Hydrologic Verification System Requirements team.
Julie Demargne (OHD/HSMB) - lead
Mary Mullusky (OCWWS/HSD)
Kevin Werner (CBRFC)
Larry Lowe (ABRFC verification focal point)
James Coe (APRFC verification focal point)
Brenda Alcorn (CBRFC verification focal point)
Alan Takamoto (CNRFC verification focal point)
Kai Roth (LMRFC verification focal point)
Bill Marosi (MARFC verification focal point and NWSEO representative)
Julie Meyer (MBRFC verification focal point)
Mike DeWeese (NCRFC verification focal point)
Rob Shedd (NERFC verification focal point)
Joe Intermill (NWRFC verification focal point)
Tom Adams (OHRFC verification focal point)
Christine McGehee (SERFC verification focal point)
Greg Waller (WGRFC verification focal point)
Verification software experts in OHD will serve as technical advisors to the team:
James Brown (OHD/HSMB)
Yuqiong Liu (OHD/HSMB)
Hank Herr (OHD/HSEB)
July 2007 – Charter Team
August 14-16 2007 – RFC verification workshop, Salt Lake City, UT
November 2007 – Kickoff team meeting to review team charter and deliverables.
December 2007 – February 2008 - Discuss verification prerequisites (e.g.archiving), identify key users, coordinate case studies, identify verification questions to address, and work on exercises with IVP and EVS software.
March – November 2008 - Member led meetings demonstrating results of IVP or EVS software for RFC verification case studies and listing required software enhancements
November 2008 – Conduct a second verification workshop
January 2009 – Interim report (including the RFC case studies reports) due
February – September 2009 - Develop standardized verification strategies for chosen user groups
September 2009 – Final Report due