Chapter 1. System Administration

Table of Contents

1. System Overview
1.1. Name and Event Server
1.2. Data Ingest Server
1.3. Data Request Server
1.4. Transmission Server
1.5. Client Programs
2. Configuring AvnFPS
2.1. Files in etc
2.2. TAF configuration files
2.3. X resources configuration files
2.4. Climatological Data
3. Starting and Stopping AvnFPS
3.1. Starting AvnFPS Servers
3.2. Stopping AvnFPS Servers
4. Avnsetup
4.1. Editing Monitoring Rules
4.2. Editing TAF Site Information
4.3. Editing TAF Site Templates
4.4. Editing TAF Product Definition
4.5. Creating Triggers
4.6. Using Text Editor
5. Configuring IFPS to send gridded data to AvnFPS
6. Logging
7. Troubleshooting
7.1. Overall Server Health
7.2. Log Files
7.3. Data Ingest
7.4. Climatology Files
7.5. Product Transmission

This chapter is intended for those who will install, configure and maintain AvnFPS.

1. System Overview

This section describes technical details of AvnFPS.

AvnFPS is implemented with a series of server and client processes that communicate via TCP sockets. The servers run on hosts where the data are readily available. Clients run on workstations which do not need direct access to the data. The client may be located on a non-AWIPS system and does not have to be within the same network segment. The data request server (avndrs) is responsible for delivering data to clients. The data ingest server (avndis) monitors data received by AWIPS and stores them in a modified format in the AvnFPS database. A notification system (avnserver) is responsible for alerting the clients when new data are received. The AvnFPS database can be distributed among several hosts.

AvnFPS is written in the Python programming language [Python]. Remote communication features are implemented with the help of Python Remote Objects (PYRO). PYRO [Pyro] is an advanced and powerful Distributed Object Technology system, written entirely in Python, which resembles Java's Remote Method Invocation (RMI). TAF and METAR decoders utilize the powerful and flexible Toy Parser Generator, [TPG], also a Python module. Graphical interfaces are written in Tkinter. The AvnFPS distribution comes with its own version of the above tools, and is independent of AWIPS COTS.

The standard AvnFPS configuration is shown below.

Data flow diagram

Data flow diagram

[Note]Note

The above diagram shows servers running on px2, however, any Red Hat™ host on a AWIPS network would do. The standard installation puts the servers on px2.

1.1. Name and Event Server

The avnserver process stores the names and locations of all objects used by AvnFPS servers, and it provides these names and locations to all other AvnFPS processes. Additionally, avnserver other task is to provide a notification mechanism between servers and clients. Thus, avnserver combines PYRO's name and event servers. One and only one instance of avnserver should run within a WFO cluster. A client first connects to avnserver to get information about the locations of the data request server and to subscribe for notifications. Client processes do not have to run on hosts at the same WFO as the data servers although this feature is generally not used. An access list of valid hosts, maintained in a configuration file, is used to provide host based access control. avnserver must be running before any other AvnFPS servers or clients can be successfully started.

The application avninit is used to launch AvnFPS servers and in a certain order. Users typically do not have to be concerned about the servers. It is avninit's job to ensure that all servers are running and restart them if they fail. However, command argument details for each server are provided here for their usefulness in troubleshooting, should it be necessary to start them by hand.

The avnserver program accepts the following command line arguments:

avnserver [-d] -n host

where host is the host name where the process will run. Obviously, this value could be retrieved from the operating system. This argument, however, is needed to properly handle failover situations. The optional flag -d tells the program to not detach itself from the terminal session, in other words, to run in the foreground.

1.2. Data Ingest Server

An instance of avndis monitors AWIPS data directories using gamin [Gamin]. avndis is a threaded application: for each data source, there is a dedicated thread that processes the data. Which server instance processes which data is determined by the server configuration file etc/server.cfg. avndis receives a notification from the gamin server when a file in a monitored directory is created or modified. The server, or parent process, then passes the file name to appropriate thread. The thread processes the file's content, writes data to AvnFPS database and returns information back to the parent process. The parent process then sends notification to avnserver. A client that subscribed to the server will then receive this notification. Currently implemented threads are:

textProcesses text products: TAFs, METARs and CCFPs.
ltgProcesses lightning data.
rltgProcesses regression-based lightning forecasts data.
llwsCalculates Low Level Wind Shear values. The data sources are radar and profiler VWP data and METAR observations.
fileMonitors files written by an external program in a format that can be processed directly by AvnFPS clients. Currently used to process IFPS grids.
guidMonitors directories where guidance sources arrive over the AWIPS WAN. Typically, BUFR MOS and raw model output from NCEP's NAM-WRF model.

Every 30 seconds avndis sends an ALIVE message that is used by clients to monitor the state of AvnFPS servers and to alert users if something goes wrong.

The program accepts the following arguments:

avndis [-d] -n host

where host is the host name where the process will run. The optional flag -d tells the program not to detach itself from the terminal session (i.e., to run in the foreground). See Section 3.1: “Starting AvnFPS Servers” for an example of ps listing.

1.3. Data Request Server

avndrs is mainly responsible for providing data to AvnFPS clients. The data source is the AvnFPS database, however, it can also directly access AWIPS files such as NetCDF data in the /data/fxa tree. The other function of avndrs is to write forecasts prepared by the clients to a queue where they are then processed by the transmission server avnxs. Each instance of avndrs is capable of processing all data sources.

Every 30 seconds avndrs sends an ALIVE message that is used by clients to monitor state of AvnFPS servers and alert users when something goes wrong.

The program accepts the following arguments:

avndrs [-d] -n host

where host is the host name where the process is running. The optional flag -d tells the program not to detach itself from the terminal session (i.e., to run in the foreground). See Section 3.1: “Starting AvnFPS Servers” for an example of ps listing.

1.4. Transmission Server

The transmission server, avnxs, provides a bridge between AvnFPS and the AWIPS transmission system. avnxs supports delayed transmissions, i. e., issuing forecasts that have been prepared by the forecaster before the transmission window opens.

Data flow diagram: Transmission Server

Data flow diagram: Transmission Server

Forecasts prepared by the AvnFPS forecast editor are written to the directory xmit/pending as text files with a particular naming convention. The filename is then used to determine the disposition of the product. Files in the xmit/pending directory are named as follows:

fid-ccccnnnxxx-wmoid-wfoid-ymmddhhmm-typ-tttttttttt

where

fid:

Forecaster ID Number

ccccnnnxxx:

AWIPS id, argument to handleOUP.pl

wmoid:

WMO Header

wfoid:

WMO ID if the WFO issuing the product

yymmddhhmm:

Full Timestamp, as year (modulo 100), month (Jan=1), day of the month, UTC hour and minute

typ:

Forecast Type and Version, one of the following

___Routine Forecast
RRXDelayed Forecast
AAXAmendment
CCXCorrection

where X is the version: A through Z.

tttttttttt:

Earliest transmission time, in Unix seconds. For routine forecasts this is the start of the transmission window or an arbitrary time set by the forecaster. For other forecasts this is the time the forecaster presses the Send button.

avnxs monitors files in the xmit/pending directory to the current system clock. The requested transmission time, Txmit, and the creation time, Tpost, are retrieved from the filename and from the OS, respectively. If the current time Tcur is within range: Txmit < Tcur < Tpost + N hours, avnxs will call an AWIPS program (currently handleOUP.pl) to send the forecast out to the WAN. N, the number of hours before a pending file is considered 'old', is now a configurable item in etc/xmit.cfg file. If the forecast file is too old, it will be moved to xmit/bad directory. The value of N (usually 3 hours) is set in transmission server configuration file. Files that do not match transmission file format are discarded. avnxs checks for the return code from handleOUP.pl. The status is written to a file xmit/DoW, where DoW is a 3-letter abbreviation for the Day Of the Week. Here is a sample entry from xmit/DoW:

SUCCESS002-KPBZTWB072-FRUS41-KPBZ-0410270100-___-1098840000

The first word is either SUCCESS or FAILnnn, and nnn is the error code returned by handleOUP.pl. If handleOUP.pl fails to transmit the forecast, the file will be moved to xmit/bad. The transmission status will be announced via avnserver to all client GUIs, see Section 4, “Forecast Transmission”.

The program accepts the following arguments:

avnxs [-d] -n host

where host is the host name where the process is running. The optional flag -d tells the program not to detach itself from the terminal session (i.e., to run in the foreground). See Section 3.1: “Starting AvnFPS Servers” for an example of the ps listing.

The transmission server also collects information on forecasts sent by the office for verification purposes. A special product is send periodically to the Performance Branch within OCWWS at NWS Headquarters that contains information that identifies which forecaster is responsible for which forecast. A configuration file etc/xmit.cfg specifies WMO and AWIPS headers and the frequency of transmission.

1.5. Client Programs

AvnWatch is the main client application. It provides forecast preparation and monitoring functionality. Avnclimate is a set of GUIs accessing archived climatological data. Current functionality provides display of historical data and statistics that can help by producing a short-term forecast based on current weather conditions. These applications are described in much greater detail in Chapter 2.