Survey Results

From: Subject: Survey Results Date: Wed, 8 Aug 2007 11:52:29 -0400 MIME-Version: 1.0 Content-Type: multipart/related; type="text/html"; boundary="----=_NextPart_000_0000_01C7D9B2.98D15450" X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3138 This is a multi-part message in MIME format. ------=_NextPart_000_0000_01C7D9B2.98D15450 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Location: https://ocwws.weather.gov/intranet/survey/results.php?sid=58 Survey Results

OCWWS Survey

Survey Results

[ Results=20 as Table [?]=20 ] [ Export=20 Results to CSV [?]=20 ]

Results for Survey: Hydrology Forecast = Verification=20 Survey for RFC Workshop

=20 Questions marked with a [*]=20 were required.

=20 [?]=20

Survey Time Stats

Average Completion Time: 22min 19sec = (Min: 0min=20 0sec, Max: 79min 40sec)
Average Time before Quit: 25min 42sec =

This survey is designed to assist in = tailoring=20 NWS training for hydrologic forecast verification. This survey = should be=20 completed by all participants for the COMET advanced hydro science = course=20 as well as the RFC verification workshop. Individual survey = responses will=20 be anonymous. No personnel, either within the NWS or connected = with this=20 project, will have access to the individual survey responses. They = will=20 have access only to summaries of the survey = responses.

Finally,=20 please note that this survey is not a test of your intelligence or = knowledge about forecasting or verification. In some cases, it = asks for=20 your perspective about tools or approaches that have not yet been=20 implemented. Your responses are important because you are among = the few=20 professional active in operational hydrologic forecasting in the = U.S. Your=20 responses will be important in devising training, procedures, and = tools=20 for verifying hydrologic forecasts in ways that can assist = forecasters as=20 they are generating forecasts.

Section I. General = Information

In this=20 section, we ask some general questions that allow us to classify = the=20 survey results based on forecasting duties, forecaster background, = and=20 years of service.

1. Which statement best describes = your=20 hydrologic forecasting duties?
Please check only one=20 statement.

I am a full time hydrologic forecaster. I perform = hydrologic=20 forecasts on daily basis - 8 3D""=20

=20 50.00% I am a full time hydrometeorology forecaster (HAS) = forecaster - 1 3D""=20

=20 6.25% I supervise a group of forecasters - 3 3D""=20

=20 18.75% Other, Please specify below - 4 3D""=20

=20 25.00% Total Answers - 16

2. Other (from question 1): =

Total Answers - 4 [=20 View Answers ]

3. Which statement best describes = your=20 formal educational training?
Please check only one = statement.

Hydrologist - 4 3D""=20

=20 25.00% Meteorologist - 5 3D""=20

=20 31.25% Environmental Scientist/Engineer - 1 3D""=20

=20 6.25% Agricultural engineer - 0 3D""=20

=20 0.00% Civil Engineer - 3 3D""=20

=20 18.75% Other, Please specify below - 3 3D""=20

=20 18.75% Total Answers - 16

4. Other (from question 3): =

Total Answers - 3 [=20 View Answers ]

5. Please indicate the number of = years you=20 have been performing hydrologic forecasting duties.
Please = check one=20 statement only.

Less than one year - 0 3D""=20

=20 0.00% 2 to 5 years - 1 3D""=20

=20 6.25% 5 to 10 years - 3 3D""=20

=20 18.75% More than 10 years - 12 3D""=20

=20 75.00% Total Answers - 16

6. After you perform stream flow = forecast,=20 your stream flow forecasts are then:
Please check all that = apply.=20

Delivered directly to WFOs - 7 3D""=20

=20 13.46% Quality controlled at the RFC and then delivered directly = to=20 WFOs - 11 3D""=20

=20 21.15% Used by other forecasters for downstream forecasting - 6 3D""=20

=20 11.54% Published directly on the web - 13 3D""=20

=20 25.00% Used to write flood warning statements by WFO = personnel - 15 3D""=20

=20 28.85% Total Answers - 52

Section II. Deterministic Streamflow=20 forecasts

In this section we attempt to identify current=20 forecasting practices and to gauge your opinion about the = forecasting=20 tools. Please answer all questions with respect to deterministic=20 Streamflow forecasts only.

7. Do you perform hydrologic=20 forecasting?
Please check only one statement.

Yes, go to question 8 - 15 3D""=20

=20 93.75% No. Thank you for your participation. Please select = 'Finish' to=20 exit the survey. - 1 3D""=20

=20 6.25% Total Answers - 16

8. Please indicate the frequency = with which=20 you perform hydrologic forecasting.
Please check only one=20 statement.

More than once a week - 12 3D""=20

=20 80.00% About once a week - 2 3D""=20

=20 13.33% About once a month - 1 3D""=20

=20 6.67% Rarely (several times a year) - 0 3D""=20

=20 0.00% Total Answers - 15

9. Do you perform forecasts = for:
Please=20 check all that apply.

Complete forecast groups - 8 3D""=20

=20 29.63% Selected basins from different forecast groups - 4 3D""=20

=20 14.81% Both complete forecast groups and selected basins from = different=20 forecast groups - 8 3D""=20

=20 29.63% My forecast points change based on needs - 7 3D""=20

=20 25.93% Total Answers - 27

10. When conducting realtime = forecasting,=20 do you perform modifications (MODS)?
Please check only one=20 statement.

Yes, go to question 11 - 15 3D""=20

=20 100.00% No, go to question 28 - 0 3D""=20

=20 0.00% Total Answers - 15

11. Which types of MODS do you = usually=20 perform?
Please check all that apply.

MAP - 13 3D""=20

=20 14.29% Q input - 6 3D""=20

=20 6.59% Q output - 8 3D""=20

=20 8.79% MAT - 6 3D""=20

=20 6.59% UH - 12 3D""=20

=20 13.19% Baseflow - 10 3D""=20

=20 10.99% Soil Reservoir Capacities - 8 3D""=20

=20 8.79% Initial model states (carry-over) of the hydrologic model = used=20 by your RFC - 6 3D""=20

=20 6.59% SWE - 7 3D""=20

=20 7.69% Snow model melt factor - 7 3D""=20

=20 7.69% Others, Please specify below - 8 3D""=20

=20 8.79% Total Answers - 91

12. Others (from question 11): =

Total Answers - 12 [=20 View Answers ]

Whether you perform MODS or not, for = each of the=20 following types of modification, please indicate your likely = action if=20 observation and model output are not close enough. Please check = only one=20 choice for each Modification.

13. MAP

Most Likely - 10 3D""=20

=20 71.43% Will perform only if other MODS were not satisfactory - 4 3D""=20

=20 28.57% Never - 0 3D""=20

=20 0.00% Total Answers - 14

14. Q input

Most Likely - 1 3D""=20

=20 7.69% Will perform only if other MODS were not satisfactory - 9 3D""=20

=20 69.23% Never - 3 3D""=20

=20 23.08% Total Answers - 13

15. Q output

Most Likely - 3 3D""=20

=20 21.43% Will perform only if other MODS were not satisfactory - 8 3D""=20

=20 57.14% Never - 3 3D""=20

=20 21.43% Total Answers - 14

16. MAT

Most Likely - 4 3D""=20

=20 26.67% Will perform only if other MODS were not satisfactory - 4 3D""=20

=20 26.67% Never - 7 3D""=20

=20 46.67% Total Answers - 15

17. UH

Most Likely - 7 3D""=20

=20 46.67% Will perform only if other MODS were not satisfactory - 6 3D""=20

=20 40.00% Never - 2 3D""=20

=20 13.33% Total Answers - 15

18. Baseflow

Most Likely - 8 3D""=20

=20 53.33% Will perform only if other MODS were not satisfactory - 3 3D""=20

=20 20.00% Never - 4 3D""=20

=20 26.67% Total Answers - 15

19. Soil Reservoir Capacities =

Most Likely - 5 3D""=20

=20 33.33% Will perform only if other MODS were not satisfactory - 5 3D""=20

=20 33.33% Never - 5 3D""=20

=20 33.33% Total Answers - 15

20. Model States (Carryover) =

Most Likely - 3 3D""=20

=20 20.00% Will perform only if other MODS were not satisfactory - 8 3D""=20

=20 53.33% Never - 4 3D""=20

=20 26.67% Total Answers - 15

21. SWE

Most Likely - 4 3D""=20

=20 26.67% Will perform only if other MODS were not satisfactory - 7 3D""=20

=20 46.67% Never - 4 3D""=20

=20 26.67% Total Answers - 15

22. Snow Model Melt Factor

Most Likely - 5 3D""=20

=20 33.33% Will perform only if other MODS were not satisfactory - 5 3D""=20

=20 33.33% Never - 5 3D""=20

=20 33.33% Total Answers - 15

23. Others (specified in Question = 12)=20

Most Likely - 6 3D""=20

=20 50.00% Will perform only if other MODS were not satisfactory - 4 3D""=20

=20 33.33% Never - 2 3D""=20

=20 16.67% Total Answers - 12

24. Please select the three most = important=20 factors affecting your decision to perform MODS. (Select only = three)=20

Recent model performance (Observed vs. Predicted) - 15 3D""=20

=20 30.61% Information from observers in the field - 5 3D""=20

=20 10.20% Long term model performance for the given basin/forecast=20 point - 2 3D""=20

=20 4.08% Performance of past modifications during similar = meteorological=20 events - 5 3D""=20

=20 10.20% My personal knowledge and familiarity with the basin - 10 3D""=20

=20 20.41% My personal knowledge and familiarity with observation=20 network - 4 3D""=20

=20 8.16% The magnitude of observations (possibility of flood) - 3 3D""=20

=20 6.12% Mass conservation (Water Balance) - 5 3D""=20

=20 10.20% Other (Please describe below) - 0 3D""=20

=20 0.00% Total Answers - 49

25. Other (from question 24) =

Total Answers - 1 [=20 View Answers ]

26. What do you think is an = acceptable=20 number of iterations for MODS when performing realtime forecast = during=20 flood hazard for an individual segment?
Please check only one=20 statement.

1-3 Iterations - 8 3D""=20

=20 53.33% 4-6 Iterations - 5 3D""=20

=20 33.33% More than 6 iterations - 2 3D""=20

=20 13.33% Total Answers - 15

27. What do you think is an = acceptable=20 number of iterations for MODS when performing realtime forecast = during=20 non-hazard periods?
Please check only one statement.

1-3 Iterations - 10 3D""=20

=20 66.67% 4-6 Iterations - 2 3D""=20

=20 13.33% More than 6 iterations - 3 3D""=20

=20 20.00% Total Answers - 15

Section III. Uncertainty in = Deterministic=20 Forecasts

In this section, we would like to identify, based = on your=20 experience, the main sources of uncertainties and errors in = deterministic=20 forecasts.

Please rate, based on your experience, = the=20 importance of matching observations and predictions for each of = the=20 following hydrograph elements.
For each hydrograph = characteristic,=20 check only one choice. If you select =93Important=92 for a = characteristic,=20 please also note the conditions when it is = important.

28. Baseflow

Important - 10 3D""=20

=20 66.67% Not Important - 5 3D""=20

=20 33.33% Total Answers - 15

29. If you marked = =91Important=92,
Please=20 note the conditions when it is important

Total Answers - 10 [=20 View Answers ]

30. Shape of rising limb

Important - 15 3D""=20

=20 100.00% Not Important - 0 3D""=20

=20 0.00% Total Answers - 15

31. If you marked = =91Important=92,
Please=20 note the conditions when it is important

Total Answers - 15 [=20 View Answers ]

32. Peakflow discharge

Important - 14 3D""=20

=20 93.33% Not Important - 1 3D""=20

=20 6.67% Total Answers - 15

33. If you marked = =91Important=92,
Please=20 note the conditions when it is important

Total Answers - 13 [=20 View Answers ]

34. Shape of recession limb =

Important - 13 3D""=20

=20 86.67% Not Important - 2 3D""=20

=20 13.33% Total Answers - 15

35. If you marked = =91Important=92,
Please=20 note the conditions when it is important

Total Answers - 12 [=20 View Answers ]

36. Total volume

Important - 13 3D""=20

=20 86.67% Not Important - 2 3D""=20

=20 13.33% Total Answers - 15

37. If you marked = =91Important=92,
Please=20 note the conditions when it is important

Total Answers - 12 [=20 View Answers ]

38. Individual ordinates of = hydrograph=20

Important - 6 3D""=20

=20 40.00% Not Important - 9 3D""=20

=20 60.00% Total Answers - 15

39. If you marked = =91Important=92,
Please=20 note the conditions when it is important

Total Answers - 7 [=20 View Answers ]

40. Low flow

Important - 9 3D""=20

=20 60.00% Not Important - 6 3D""=20

=20 40.00% Total Answers - 15

41. If you marked = =91Important=92,
Please=20 note the conditions when it is important

Total Answers - 9 [=20 View Answers ]

42. Time to peak

Important - 14 3D""=20

=20 93.33% Not Important - 1 3D""=20

=20 6.67% Total Answers - 15

43. If you marked = =91Important=92,
Please=20 note the conditions when it is important

Total Answers - 13 [=20 View Answers ]

Please rank, based on your experience, = the degree=20 each of the following may contribute to forecast error and = uncertainties.=20 As in operational cases, assume all models to have been = pre-calibrated.=20
Please check only one choice for each NWSRFS = Element.

44. Hydrology model (SAC-SMA,API) = parameters

Very Important - 8 3D""=20

=20 57.14% Moderately Important - 4 3D""=20

=20 28.57% Somewhat Important - 2 3D""=20

=20 14.29% Not Important - 0 3D""=20

=20 0.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 14

45. Hydrology model (e.g. = SAC-SMA,API)=20 structure

Very Important - 3 3D""=20

=20 20.00% Moderately Important - 4 3D""=20

=20 26.67% Somewhat Important - 5 3D""=20

=20 33.33% Not Important - 1 3D""=20

=20 6.67% Not Sure - 2 3D""=20

=20 13.33% Total Answers - 15

46. Initial conditions = (Carryover)

Very Important - 8 3D""=20

=20 53.33% Moderately Important - 4 3D""=20

=20 26.67% Somewhat Important - 3 3D""=20

=20 20.00% Not Important - 0 3D""=20

=20 0.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 15

47. Snow model (SNOW-17) = parameters

Very Important - 7 3D""=20

=20 46.67% Moderately Important - 3 3D""=20

=20 20.00% Somewhat Important - 1 3D""=20

=20 6.67% Not Important - 2 3D""=20

=20 13.33% Not Sure - 2 3D""=20

=20 13.33% Total Answers - 15

48. Snow model (SNOW-17) = structure

Very Important - 4 3D""=20

=20 26.67% Moderately Important - 5 3D""=20

=20 33.33% Somewhat Important - 2 3D""=20

=20 13.33% Not Important - 2 3D""=20

=20 13.33% Not Sure - 2 3D""=20

=20 13.33% Total Answers - 15

49. Unit Hydrograph

Very Important - 6 3D""=20

=20 40.00% Moderately Important - 7 3D""=20

=20 46.67% Somewhat Important - 2 3D""=20

=20 13.33% Not Important - 0 3D""=20

=20 0.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 15

50. Precipitation input = (Observations)=20

Very Important - 15 3D""=20

=20 100.00% Moderately Important - 0 3D""=20

=20 0.00% Somewhat Important - 0 3D""=20

=20 0.00% Not Important - 0 3D""=20

=20 0.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 15

51. Precipitation input = (Forecast)

Very Important - 12 3D""=20

=20 80.00% Moderately Important - 2 3D""=20

=20 13.33% Somewhat Important - 1 3D""=20

=20 6.67% Not Important - 0 3D""=20

=20 0.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 15

52. Temperature input = (Observations)

Very Important - 6 3D""=20

=20 40.00% Moderately Important - 4 3D""=20

=20 26.67% Somewhat Important - 2 3D""=20

=20 13.33% Not Important - 3 3D""=20

=20 20.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 15

53. Temperature input (Forecasts) =

Very Important - 6 3D""=20

=20 40.00% Moderately Important - 4 3D""=20

=20 26.67% Somewhat Important - 2 3D""=20

=20 13.33% Not Important - 3 3D""=20

=20 20.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 15

54. Upstream inflow (forecasts) =

Very Important - 7 3D""=20

=20 46.67% Moderately Important - 7 3D""=20

=20 46.67% Somewhat Important - 1 3D""=20

=20 6.67% Not Important - 0 3D""=20

=20 0.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 15

55. Upstream inflow = (observations)

Very Important - 11 3D""=20

=20 73.33% Moderately Important - 3 3D""=20

=20 20.00% Somewhat Important - 1 3D""=20

=20 6.67% Not Important - 0 3D""=20

=20 0.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 15

56. Reservoir release/level = projections=20

Very Important - 12 3D""=20

=20 80.00% Moderately Important - 2 3D""=20

=20 13.33% Somewhat Important - 1 3D""=20

=20 6.67% Not Important - 0 3D""=20

=20 0.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 15

57. Rating curves

Very Important - 11 3D""=20

=20 73.33% Moderately Important - 3 3D""=20

=20 20.00% Somewhat Important - 1 3D""=20

=20 6.67% Not Important - 0 3D""=20

=20 0.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 15

58. Routing procedures

Very Important - 8 3D""=20

=20 53.33% Moderately Important - 5 3D""=20

=20 33.33% Somewhat Important - 2 3D""=20

=20 13.33% Not Important - 0 3D""=20

=20 0.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 15

59. Recent = modifications/carry-over

Very Important - 6 3D""=20

=20 40.00% Moderately Important - 6 3D""=20

=20 40.00% Somewhat Important - 3 3D""=20

=20 20.00% Not Important - 0 3D""=20

=20 0.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 15

60. Freezing level

Very Important - 4 3D""=20

=20 26.67% Moderately Important - 1 3D""=20

=20 6.67% Somewhat Important - 1 3D""=20

=20 6.67% Not Important - 7 3D""=20

=20 46.67% Not Sure - 2 3D""=20

=20 13.33% Total Answers - 15

61. SWE

Very Important - 10 3D""=20

=20 66.67% Moderately Important - 0 3D""=20

=20 0.00% Somewhat Important - 0 3D""=20

=20 0.00% Not Important - 3 3D""=20

=20 20.00% Not Sure - 2 3D""=20

=20 13.33% Total Answers - 15

Section IV. General = Verification

62. Many forecasters informally = monitor the=20 skills of forecasts they issue. This is meant to monitor their own = skills=20 and/or the skills of the forecast system and its ability to = provide=20 accurate and reliable forecasts for different conditions. Please = check=20 only the one statement that best describes the type of forecast = skill=20 monitoring activities you perform.

I monitor forecast skills only for selected basins - 2 3D""=20

=20 14.29% I monitor forecast skills for all forecast points I am=20 responsible for - 5 3D""=20

=20 35.71% I conduct random sampling of forecast skills - 1 3D""=20

=20 7.14% In my RFC, monitoring forecast skill is performed by = management=20 and /or senior forecasters only - 1 3D""=20

=20 7.14% In my RFC, there is anecdotal or event based monitoring of = skill, but no systematic monitoring of skill - 5 3D""=20

=20 35.71% In my RFC, there is no monitoring of forecast skill - 0 3D""=20

=20 0.00% In my RFC, there is monitoring of forecast skill, but I am = not a=20 participant - 0 3D""=20

=20 0.00% Total Answers - 14

In general, forecast verification = involves=20 calculation of several performance scores that provide information = on the=20 differences between forecast and observations. There are many = possible=20 measures that can provide some information about such differences = and the=20 list below includes some of these measures. Please indicate = whether or not=20 you are familiar with these terms. Please check only one answer = for each=20 Skill Measure.

63. Root Mean Square Error

Used It - 9 3D""=20

=20 60.00% Very Familiar - 3 3D""=20

=20 20.00% Familiar - 3 3D""=20

=20 20.00% Not Familiar - 0 3D""=20

=20 0.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

64. Nash-Sutcliffe Index

Used It - 0 3D""=20

=20 0.00% Very Familiar - 0 3D""=20

=20 0.00% Familiar - 1 3D""=20

=20 6.67% Not Familiar - 14 3D""=20

=20 93.33% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

65. Visual inspection of = hydrographs

Used It - 12 3D""=20

=20 80.00% Very Familiar - 2 3D""=20

=20 13.33% Familiar - 0 3D""=20

=20 0.00% Not Familiar - 1 3D""=20

=20 6.67% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

66. Scatter plots

Used It - 7 3D""=20

=20 46.67% Very Familiar - 4 3D""=20

=20 26.67% Familiar - 4 3D""=20

=20 26.67% Not Familiar - 0 3D""=20

=20 0.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

67. Correlation coefficient =

Used It - 5 3D""=20

=20 38.46% Very Familiar - 0 3D""=20

=20 0.00% Familiar - 5 3D""=20

=20 38.46% Not Familiar - 3 3D""=20

=20 23.08% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 13

68. Critical Success Index

Used It - 0 3D""=20

=20 0.00% Very Familiar - 0 3D""=20

=20 0.00% Familiar - 6 3D""=20

=20 40.00% Not Familiar - 9 3D""=20

=20 60.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

69. Mean Absolute Error

Used It - 10 3D""=20

=20 66.67% Very Familiar - 4 3D""=20

=20 26.67% Familiar - 1 3D""=20

=20 6.67% Not Familiar - 0 3D""=20

=20 0.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

70. Gerrity Score

Used It - 0 3D""=20

=20 0.00% Very Familiar - 0 3D""=20

=20 0.00% Familiar - 1 3D""=20

=20 6.67% Not Familiar - 14 3D""=20

=20 93.33% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

71. Linear error in probability = score=20

Used It - 0 3D""=20

=20 0.00% Very Familiar - 0 3D""=20

=20 0.00% Familiar - 2 3D""=20

=20 14.29% Not Familiar - 12 3D""=20

=20 85.71% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 14

72. Discrimination Diagram

Used It - 0 3D""=20

=20 0.00% Very Familiar - 0 3D""=20

=20 0.00% Familiar - 5 3D""=20

=20 33.33% Not Familiar - 10 3D""=20

=20 66.67% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

73. Probability of detection =

Used It - 5 3D""=20

=20 33.33% Very Familiar - 3 3D""=20

=20 20.00% Familiar - 5 3D""=20

=20 33.33% Not Familiar - 2 3D""=20

=20 13.33% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

74. Brier Score

Used It - 1 3D""=20

=20 6.67% Very Familiar - 1 3D""=20

=20 6.67% Familiar - 7 3D""=20

=20 46.67% Not Familiar - 6 3D""=20

=20 40.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

75. Ranked Probability Score =

Used It - 0 3D""=20

=20 0.00% Very Familiar - 1 3D""=20

=20 6.67% Familiar - 7 3D""=20

=20 46.67% Not Familiar - 7 3D""=20

=20 46.67% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

76. Forecast Resolution

Used It - 0 3D""=20

=20 0.00% Very Familiar - 1 3D""=20

=20 6.67% Familiar - 4 3D""=20

=20 26.67% Not Familiar - 10 3D""=20

=20 66.67% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

77. Forecast Reliability

Used It - 1 3D""=20

=20 6.67% Very Familiar - 0 3D""=20

=20 0.00% Familiar - 5 3D""=20

=20 33.33% Not Familiar - 9 3D""=20

=20 60.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

78. Forecast Sharpness

Used It - 0 3D""=20

=20 0.00% Very Familiar - 0 3D""=20

=20 0.00% Familiar - 6 3D""=20

=20 42.86% Not Familiar - 8 3D""=20

=20 57.14% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 14

79. Reliability Diagram

Used It - 1 3D""=20

=20 6.67% Very Familiar - 0 3D""=20

=20 0.00% Familiar - 3 3D""=20

=20 20.00% Not Familiar - 11 3D""=20

=20 73.33% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

80. Relative Operating = Characteristic=20

Used It - 0 3D""=20

=20 0.00% Very Familiar - 0 3D""=20

=20 0.00% Familiar - 2 3D""=20

=20 13.33% Not Familiar - 13 3D""=20

=20 86.67% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

81. Correspondence Ratio

Used It - 0 3D""=20

=20 0.00% Very Familiar - 0 3D""=20

=20 0.00% Familiar - 0 3D""=20

=20 0.00% Not Familiar - 15 3D""=20

=20 100.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

82. False Alarm Ratio

Used It - 5 3D""=20

=20 33.33% Very Familiar - 5 3D""=20

=20 33.33% Familiar - 5 3D""=20

=20 33.33% Not Familiar - 0 3D""=20

=20 0.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

83. Ranked Histogram

Used It - 1 3D""=20

=20 6.67% Very Familiar - 2 3D""=20

=20 13.33% Familiar - 4 3D""=20

=20 26.67% Not Familiar - 8 3D""=20

=20 53.33% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

84. Visual inspection of = probability=20 distributions

Used It - 3 3D""=20

=20 20.00% Very Familiar - 2 3D""=20

=20 13.33% Familiar - 2 3D""=20

=20 13.33% Not Familiar - 8 3D""=20

=20 53.33% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 15

85. Forecast skill is typically = expressed=20 in terms relative to the next best alternative forecast. For the = forecasts=20 you make, please indicate what would you consider to be the next = best=20 alternative forecast. Please select only one.

Persistence - 2 3D""=20

=20 13.33% Long term mean of observations (climatology) - 2 3D""=20

=20 13.33% Trended persistence - 1 3D""=20

=20 6.67% A Raw Model (model without any forecaster = interference) - 10 3D""=20

=20 66.67% Total Answers - 15

Section V. Retrospective = Analyses

In this=20 section, a retrospective analysis study is defined as a study that = aims to=20 further understand and improve the performance of forecast system. = This=20 may include: (a) re-calibration of forecast models, (b) assessment = of=20 forecast skills at specific points, and (c) diagnostic assessments = of=20 system-wide. In the following series of question, we attempt to = gauge=20 forecaster=92s involvement as well as their assessment of the = benefit and=20 needed improvement of such studies.

86. Please check the type = in-house of post=20 event analysis studies you have performed or participated in = performing.=20
Please check all that apply

Calibration for new forecast points/basins - 9 3D""=20

=20 20.45% Model re-calibration - 9 3D""=20

=20 20.45% Deterministic forecast verification (Forecast skill=20 assessment) - 11 3D""=20

=20 25.00% Probabilistic forecast verification - 4 3D""=20

=20 9.09% Re-forecasts (Hindcasts) for assessing forecast skill - 2 3D""=20

=20 4.55% None, I personally have not done or participated in post = event=20 analysis studies - 0 3D""=20

=20 0.00% My office performs post event analysis studies - 9 3D""=20

=20 20.45% Total Answers - 44

Section VI. Ensemble Streamflow=20 Predictions

The NWSRFS includes several tools for = generating ESP=20 forecasts. As the NWS attempts to improve ESP generation and = verification=20 capabilities, we are trying to determine what tools may be useful = to=20 extend, and what type of verification statistics might be included = in=20 future verification tools

87. Do you perform ESP=20 forecasting?
Please check only one statement.

Yes, please continue with the question 87 - 10 3D""=20

=20 66.67% No . Thank you for your participation. Please select = 'Finish' to=20 exit the survey. - 5 3D""=20

=20 33.33% Total Answers - 15

88. Relative to the number of = points for=20 which you issue deterministic forecasts, what is the proportion of = points=20 for which you also issue probabilistic forecasts?
Please write = in the=20 proportion:

Total Answers - 10 [=20 View Answers ]

89. When generating ESP = forecasts, do you:=20
Please check only one statement.

Use ESPADP/ESPINIT Manager - 4 3D""=20

=20 40.00% ETSGEN Interface - 0 3D""=20

=20 0.00% Direct scripting (ens tools) - 6 3D""=20

=20 60.00% Total Answers - 10

90. To create ESP Products, do=20 you:
Please check only one statement.

Use ESPADP graphical user interface - 2 3D""=20

=20 20.00% ESPADP scripts (ens tools) - 8 3D""=20

=20 80.00% Total Answers - 10

91. In general, when generating = the=20 Exceedance plots, do you use:
Please check only one=20 statement.

Empirical distribution - 6 3D""=20

=20 75.00% Others from those available in ESPADP - 2 3D""=20

=20 25.00% Total Answers - 8

92. Have you attended formal = training on=20 ESP tools in the NWSRFS?
Please check only one = statement.

No - 5 3D""=20

=20 50.00% Yes - 5 3D""=20

=20 50.00% Total Answers - 10

As mentioned above, NWSRFS includes = several tools=20 that improve the issuance and management of probabilistic = forecasts,=20 please indicate your level of familiarity with the following=20 tools.
Please check only one column for each = Tool.

93. ENS_PRE

Use it - 3 3D""=20

=20 30.00% Had training on using - 0 3D""=20

=20 0.00% Familiar with - 3 3D""=20

=20 30.00% Never Used - 4 3D""=20

=20 40.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 10

94. ENS_POST

Use it - 1 3D""=20

=20 10.00% Had training on using - 1 3D""=20

=20 10.00% Familiar with - 3 3D""=20

=20 30.00% Never Used - 5 3D""=20

=20 50.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 10

95. ENS_PRE_CP

Use it - 2 3D""=20

=20 20.00% Had training on using - 0 3D""=20

=20 0.00% Familiar with - 3 3D""=20

=20 30.00% Never Used - 5 3D""=20

=20 50.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 10

96. ENS_POST_CP

Use it - 1 3D""=20

=20 10.00% Had training on using - 2 3D""=20

=20 20.00% Familiar with - 1 3D""=20

=20 10.00% Never Used - 6 3D""=20

=20 60.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 10

97. ETSGEN

Use it - 0 3D""=20

=20 0.00% Had training on using - 0 3D""=20

=20 0.00% Familiar with - 1 3D""=20

=20 10.00% Never Used - 9 3D""=20

=20 90.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 10

98. ESPVS

Use it - 0 3D""=20

=20 0.00% Had training on using - 0 3D""=20

=20 0.00% Familiar with - 2 3D""=20

=20 20.00% Never Used - 8 3D""=20

=20 80.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 10

99. TSTool

Use it - 0 3D""=20

=20 0.00% Had training on using - 0 3D""=20

=20 0.00% Familiar with - 1 3D""=20

=20 10.00% Never Used - 9 3D""=20

=20 90.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 10

100. ESPADP

Use it - 9 3D""=20

=20 90.00% Had training on using - 1 3D""=20

=20 10.00% Familiar with - 0 3D""=20

=20 0.00% Never Used - 0 3D""=20

=20 0.00% Not Applicable to Duties - 0 3D""=20

=20 0.00% Total Answers - 10

Originally, ESP was developed for = long-term=20 forecasts (30 days or longer). Please indicate your = agreement/disagreement=20 level with each of the following statements regarding the utility = of ESP=20 for shorter terms probabilistic forecasts.
Please check only = one column=20 for each Statement.

101. ESP forecasts are only = useful for long=20 term forecasts 1-3 month

Strongly Agree - 1 3D""=20

=20 10.00% Somehow agree - 3 3D""=20

=20 30.00% Somehow disagree - 1 3D""=20

=20 10.00% Strongly disagree - 5 3D""=20

=20 50.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 10

102. Short term ESPs (1-3 days) = from the=20 current system are appropriate for use

Strongly Agree - 1 3D""=20

=20 10.00% Somehow agree - 1 3D""=20

=20 10.00% Somehow disagree - 0 3D""=20

=20 0.00% Strongly disagree - 4 3D""=20

=20 40.00% Not Sure - 4 3D""=20

=20 40.00% Total Answers - 10

103. Short term ESPs can be = issued only if=20 traces are weighed using other information

Strongly Agree - 1 3D""=20

=20 10.00% Somehow agree - 0 3D""=20

=20 0.00% Somehow disagree - 0 3D""=20

=20 0.00% Strongly disagree - 1 3D""=20

=20 10.00% Not Sure - 8 3D""=20

=20 80.00% Total Answers - 10

104. Short term ESPs can only be = issued if=20 uncertainty in initial condition is included in the generation = process=20

Strongly Agree - 0 3D""=20

=20 0.00% Somehow agree - 3 3D""=20

=20 30.00% Somehow disagree - 2 3D""=20

=20 20.00% Strongly disagree - 1 3D""=20

=20 10.00% Not Sure - 4 3D""=20

=20 40.00% Total Answers - 10

105. Probabilistic forecasts will = replace=20 deterministic forecasts for both short and long term predictions =

Strongly Agree - 3 3D""=20

=20 30.00% Somehow agree - 2 3D""=20

=20 20.00% Somehow disagree - 2 3D""=20

=20 20.00% Strongly disagree - 3 3D""=20

=20 30.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 10

106. Future implementations of = ESP should=20 incorporate other sources of uncertainty in addition to = uncertainty in=20 future precipitation and temperature

Strongly Agree - 5 3D""=20

=20 50.00% Somehow agree - 4 3D""=20

=20 40.00% Somehow disagree - 1 3D""=20

=20 10.00% Strongly disagree - 0 3D""=20

=20 0.00% Not Sure - 0 3D""=20

=20 0.00% Total Answers - 10

107. Incorporating other sources = of=20 uncertainty in probabilistic forecasts will significantly = complicate the=20 operational procedure use to produce ESP

Strongly Agree - 5 3D""=20

=20 50.00% Somehow agree - 3 3D""=20

=20 30.00% Somehow disagree - 0 3D""=20

=20 0.00% Strongly disagree - 1 3D""=20

=20 10.00% Not Sure - 1 3D""=20

=20 10.00% Total Answers - 10

108. Including other sources of = uncertainty=20 will adversely affect the users=92 ability to interpret ESP = products

Strongly Agree - 1 3D""=20

=20 10.00% Somehow agree - 2 3D""=20

=20 20.00% Somehow disagree - 2 3D""=20

=20 20.00% Strongly disagree - 2 3D""=20

=20 20.00% Not Sure - 3 3D""=20

=20 30.00% Total Answers - 10

109. Incorporating other sources = of=20 uncertainty will
enhance the usage of ESP products

Strongly Agree - 4 3D""=20

=20 40.00% Somehow agree - 4 3D""=20

=20 40.00% Somehow disagree - 0 3D""=20

=20 0.00% Strongly disagree - 0 3D""=20

=20 0.00% Not Sure - 2 3D""=20

=20 20.00% Total Answers - 10

Section VII. Verification of = Probabilistic=20 Forecasts

As mentioned in the introduction to this survey, = the=20 ultimate goal of this survey is to assist the NWS in determining = the most=20 appropriate means to allow forecasters to use various measures of = the=20 skill of probabilistic forecasts in realtime modeling. In the next = series=20 of questions, we attempt to identify, with the help of = forecasters, a set=20 of guidelines for such means.

A robust verification of probabilistic = forecasts=20 requires information about past forecasts. From your own = perspective,=20 please indicate the level of importance you would assign to each = of the=20 information listed below.
Please check only one column for each = NWSRFS=20 Element.

110. Complete retrospective = performance of=20 ESP for all possible forecasts (Full Hindcast)

Very Important - 3 3D""=20

=20 30.00% Moderately Important - 3 3D""=20

=20 30.00% Somewhat Important - 2 3D""=20

=20 20.00% Not Important - 0 3D""=20

=20 0.00% Not Sure - 2 3D""=20

=20 20.00% Total Answers - 10

111. Skill of forecasts for the = current=20 forecast window, from previous years.

Very Important - 4 3D""=20

=20 40.00% Moderately Important - 3 3D""=20

=20 30.00% Somewhat Important - 1 3D""=20

=20 10.00% Not Important - 0 3D""=20

=20 0.00% Not Sure - 2 3D""=20

=20 20.00% Total Answers - 10

112. Skill of recent forecasts = leading up=20 to the current forecast window.

Very Important - 5 3D""=20

=20 50.00% Moderately Important - 4 3D""=20

=20 40.00% Somewhat Important - 0 3D""=20

=20 0.00% Not Important - 0 3D""=20

=20 0.00% Not Sure - 1 3D""=20

=20 10.00% Total Answers - 10

113. Evaluation of forecasts for = periods=20 having similar climatic and hydrologic conditions

Very Important - 4 3D""=20

=20 40.00% Moderately Important - 3 3D""=20

=20 30.00% Somewhat Important - 2 3D""=20

=20 20.00% Not Important - 0 3D""=20

=20 0.00% Not Sure - 1 3D""=20

=20 10.00% Total Answers - 10

US Dept of=20 Commerce
National Oceanic = and=20 Atmospheric Administration
National Weather = Service
Office of=20 Climate, Water & Weather Services
1325 East West = Highway
Silver=20 Spring, MD 20910
Page Author: Michael T. Smith Disclaimer
Credits
Glossary
Privacy = Policy
About Us
Career = Opportunities Last Updated: December 31, = 1969

NOTE: The Links below will take you out = of All=20 Government Domains:
[ License=20 | Change = Log=20 | To Do=20 List | Bugs ] =
Powered=20 by UCCASS v1.06. = Copyright=20 =A9 2004 John W. Holmes, All = Rights=20 Reserved

------=_NextPart_000_0000_01C7D9B2.98D15450 Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Location: https://ocwws.weather.gov/intranet/survey/templates/Default/images/aquabar_left.gif R0lGODlhBQAQAMQAAABWxzxVdXSt7IC076DO/5nI+qnP9rrY96bU/7Xh/8bu/87///Ly8vDw8O3t 7ezs7Onp6efn5+Pj49/f397e3tvb28rKysfHx7W1tbS0tKqqqqampqKiopqamo6OjoeHhyH5BAAA AAAALAAAAAAFABAAAAU1INM4kdRA1YU9FfYB0OV1h5RxhjFpQj9tgyBlUyj+CMgfYrmzJBI2jkIR my1YrsAphRGRJCEAOw== ------=_NextPart_000_0000_01C7D9B2.98D15450 Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Location: https://ocwws.weather.gov/intranet/survey/templates/Default/images/aquabar.gif R0lGODlhAQAQAMQQAAAAAP///wBWxzxVdXSt7KzP9oS58JXH+Z3M/aXS/7jX97Xh/8bu/87//+Pj 47W1tQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEA ABAALAAAAAABABAAAAUOoPMISkEYB5IsTDM8TggAOw== ------=_NextPart_000_0000_01C7D9B2.98D15450 Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Location: https://ocwws.weather.gov/intranet/survey/templates/Default/images/aquabar_right.gif R0lGODlhBQAQAMQAAABWxzxVdXSt7IC076DO/5nI+qnP9rrY96bU/7Xh/8bu/87///Ly8vDw8Ozs 7Onp6ejo6Ofn5+Pj49/f397e3tvb28rKysfHx7W1tbS0tKqqqqampqKiopqamo6OjoeHhyH5BAAA AAAALAAAAAAFABAAAAU1oCQ2DHZVTwN8WOUcnXdBhsFlkqBr0+Bvk4JwQyEYgYgkMJGw8BSKm2QR mwVYLhOqIYk4SCEAOw== ------=_NextPart_000_0000_01C7D9B2.98D15450 Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Location: https://ocwws.weather.gov/intranet/survey/templates/Default/images/bar_left.gif R0lGODlhBQAQAMQAAABWxzxVdXSt7IC076DO/5nI+qnP9rrY96bU/7Xh/8bu/87///Ly8vDw8O3t 7ezs7Onp6efn5+Pj49/f397e3tvb28rKysfHx7W1tbS0tKqqqqampqKiopqamo6OjoeHhyH5BAAA AAAALAAAAAAFABAAAAU1INM4kdRA1YU9FfYB0OV1h5RxhjFpQj9tgyBlUyj+CMgfYrmzJBI2jkIR my1YrsAphRGRJCEAOw== ------=_NextPart_000_0000_01C7D9B2.98D15450 Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Location: https://ocwws.weather.gov/intranet/survey/templates/Default/images/bar.gif R0lGODlhAQAQAMQQAAAAAP///wBWxzxVdXSt7KzP9oS58JXH+Z3M/aXS/7jX97Xh/8bu/87//+Pj 47W1tQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEA ABAALAAAAAABABAAAAUOoPMISkEYB5IsTDM8TggAOw== ------=_NextPart_000_0000_01C7D9B2.98D15450 Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Location: https://ocwws.weather.gov/intranet/survey/templates/Default/images/bar_right.gif R0lGODlhBQAQAMQAAABWxzxVdXSt7IC076DO/5nI+qnP9rrY96bU/7Xh/8bu/87///Ly8vDw8Ozs 7Onp6ejo6Ofn5+Pj49/f397e3tvb28rKysfHx7W1tbS0tKqqqqampqKiopqamo6OjoeHhyH5BAAA AAAALAAAAAAFABAAAAU1oCQ2DHZVTwN8WOUcnXdBhsFlkqBr0+Bvk4JwQyEYgYgkMJGw8BSKm2QR mwVYLhOqIYk4SCEAOw== ------=_NextPart_000_0000_01C7D9B2.98D15450 Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Location: https://ocwws.weather.gov/intranet/survey/templates/Default/images/background.gif R0lGODlhYwBjAIAAANnZ2f///yH5BAAAAAAALAAAAABjAGMAAAL/jI+py+0PYwMgUFuvzhz7/XXg KJbhSaJmyq6uCrfxK9X2jefy/vLz7wv2hkCisHjKKZfM2/FpjCKhUyn1Km1qt9yqF/u1isNksK+L TtfMY3a5DX/LZeq6XeHOx/XzvR95F2jHR/hX2IcIJriIduhoCJkYecRYqfUomYm5GWnpqcSpOTkq Wvp5ahOqSrpaSocK+9A6y1rrShObu0B728uLqRuM8GtL7EsqnGy8XNyMmCzMfDwtTQmtW+2cTU10 ja0Nzi1+6J27fR5uXB6LPp7+/rMO204PHyqPWu++r47/qQ/QHjJ/ngLyEyiOYEGEBhvGUWjJ4cGJ wCAykoiRYRaLKYsyUtSYiWNHkB5LmhApyOTHlYpQ3lEJk2QFly9lsrR5j2admDd78igAADs= ------=_NextPart_000_0000_01C7D9B2.98D15450 Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Location: https://ocwws.weather.gov/intranet/survey/templates/Default/images/box_bg.gif R0lGODlhAQAXAMQAAOTl5ejo6d7e3+7u7+Dh4djY2eLi4+rr6qKio+vt7O/w8NLT09TU1tbW19HR 0/Ly8tzd3b/AwObn59ra2/Dx8f///wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAAA AAAALAAAAAABABcAAAUTIFJVzsI0xQQJhAFIwZEMCvVEIQA7 ------=_NextPart_000_0000_01C7D9B2.98D15450 Content-Type: text/css; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Location: https://ocwws.weather.gov/intranet/survey/templates/Default/style.css BODY { FONT-SIZE: 12px; COLOR: #000000; FONT-FAMILY: = Arial,Helvetica,sans-serif; BACKGROUND-COLOR: #f3f3f3 } TABLE { FONT-SIZE: 12px; FONT-FAMILY: Arial,Helvetica,sans-serif } TR { FONT-SIZE: 12px; FONT-FAMILY: Arial,Helvetica,sans-serif } TD { FONT-SIZE: 12px; FONT-FAMILY: Arial,Helvetica,sans-serif } TH { FONT-SIZE: 12px; FONT-FAMILY: Arial,Helvetica,sans-serif } .bordered_table { BORDER-RIGHT: #bdbdbd 1px solid; BORDER-TOP: #bdbdbd 1px solid; = FONT-SIZE: 12px; BORDER-LEFT: #bdbdbd 1px solid; BORDER-BOTTOM: #bdbdbd = 1px solid; FONT-FAMILY: Arial,Helvetica,sans-serif; BACKGROUND-COLOR: = #f9f9f9 } .grayboxheader { FONT-WEIGHT: bold; FONT-SIZE: 12px; COLOR: #7e7d7e; FONT-FAMILY: Arial, = Helvetica, sans-serif; TEXT-ALIGN: center } .whitebox { FONT-WEIGHT: bold; FONT-SIZE: 12px; COLOR: #000000; FONT-FAMILY: Arial, = Helvetica, sans-serif; BACKGROUND-COLOR: #ffffff } .indented_cell { MARGIN-TOP: 5px; FONT-SIZE: 12px; MARGIN-BOTTOM: 5px; MARGIN-LEFT: = 15px; FONT-FAMILY: Arial, Helvetica, sans-serif } .copyright { FONT-SIZE: 12px; COLOR: #000000; FONT-FAMILY: Arial, Helvetica, = sans-serif; TEXT-ALIGN: center } .example { FONT-WEIGHT: normal; FONT-SIZE: smaller; FONT-STYLE: italic } .message { FONT-WEIGHT: bold; COLOR: blue } .grayboxfont { FONT-WEIGHT: normal; FONT-SIZE: 12px; COLOR: #424142; FONT-FAMILY: = Arial, Helvetica, sans-serif } .question { FONT-WEIGHT: bold; FONT-SIZE: 12px; COLOR: #000000; FONT-FAMILY: Arial, = Helvetica, sans-serif } .percentage { FONT-WEIGHT: normal; FONT-SIZE: 10px; COLOR: #000000; FONT-FAMILY: = Arial, Helvetica, sans-serif } .error { FONT-WEIGHT: bold; FONT-SIZE: 16px; COLOR: #cb0404; FONT-FAMILY: Arial, = Helvetica, sans-serif } ------=_NextPart_000_0000_01C7D9B2.98D15450--