make printable	new:	see also:
	v2.1: - call as many trendPlotter sessions in parallel as your host can crunch - navigation bars can be created by webNavBar	all trendPlotter config files operational HealthCheck job files trendPlotter configuration trendPlotter tool qc1Plotter web interface
topics: trending archive | history plots | maintenance | parallel execution

trendPlotter: how to operate

General structure of trending plot archive. The set of trending plots spans a 2D matrix: for each report, there is a set of HISTORY plots, and the current HEALTH plot. Let's assume there are four different reports, and there are HISTORY plots since 2005. Then at the end of 2006 the trending report archive has the following reports:

Obviously this information is steadily growing, and the challenge is to present this in an easy-to-use and intuitive way. Over the years, two different views have evolved: the Health Check Monitor, and the interface to the history plots. Originally this was on the tutorial pages. With the trendplotter tool, the HISTORY interface is now part of the Health Check pages. These two views are called primary views in the following.

The Health Check monitor offers a view to all HEALTH reports. It is the main portal from Paranal to the trending reports and web pages:

The QC tutorial page focusses on a particular report, say report1. It has a link to all HISTORY plots and the HEALTH plot for that given report. It combines all information about the report, including background information about the parameters (what is trended, why is it trended, which instrument component is controlled by the report etc.):

The two primary views give direct access to the reports marked in color ('direct' meaning: through a vertical navigation bar, plus possibly a history navigation bar).

Creating all HISTORY plots for a given report. The option -A (all) creates an executable file with all trendPlotter calls required to create the historical report set. Let's assume that you need to create all historical BIAS reports, starting from 2004-04, up to 2006-10. Then calling 'trendPlotter -r BIAS -A' creates a file $TMP_DIR/list_history containing all calls

trendPlotter -r BIAS -t HISTORY -s 2004-04 -f
trendPlotter -r BIAS -t HISTORY -s 2004-07 -f
trendPlotter -r BIAS -t HISTORY -s 2004-10 -f
...
trendPlotter -r BIAS -t HISTORY -s 2006-10 -f

Important: You need to edit this file to remove the option -f, if you want to do tests and not publish the results.
In any case, the file $TMP_DIR/list_history won't be executed directly but needs to be invoked by the user.

Creating all HISTORY navigation bars. The option -N (navigation bars) creates or refreshes all HISTORY navigation bars for a specified report. This is necessary whenever a new time period is added to the historical record, or of course if a new report has been defined. Calling 'trendPlotter -r BIAS -N [-f]' will create all history navigation bars and publish them if option -f is set.

The START_DATE for a report is configured in config.tp_<report>.

Creating the latest HISTORY plot. Calling 'trendPlotter -r BIAS -t HISTORY [-f]', i.e. without explicit start date, will let the tool determine the latest ("current") history plot. The tool evaluates the current date, and the configured RANGE. This is useful for updating the current HISTORY plot regularly by a cronjob.

Maintenance

How to maintain HEALTH plots. Write all HEALTH CHECK jobs which can be called by qc1Parser into a job file $DFO_JOB_DIR/JOBS_HEALTH:

trendPlotter -r <report> -t HEALTH -f

Then, they will be updated incrementally by qc1Parser (to be called once per hour in incremental mode, by a cronjob) when new opslog values have been found, as a compromise between the competing goals completeness and up-to-date-ness.

In order to be always "reasonably" up-to-date and complete, write all HEALTH CHECK jobs in a second jobs file $DFO_JOB_DIR/JOBS_TREND and call this explicitly in your cronjob file at least every 12 hours.

Find the current operational JOBS_HEALTH and JOBS_TREND for FLAMES/GIRAFFE here.

Using the AVG option FIXED requires special attention since this is considered a temporary measure if e.g. a parameter has jumped suddenly and the new AVG value cannot be properly calculated by MEAN nor MEDIAN. Using FIXED solves this issue but at the price that you may loose trends in the data. Eventually (once you have accumulated enough new data points) you may want to go back to either MEAN or MEDIAN.

How to maintain navigation bars. You will also need to refresh all HISTORY navigation bars regularly, since every 60/90/180/365 days a new HISTORY plot will be added to the set by JOBS_HISTORY, and as long as the navigation bars are not updated, that plot is not linked to the other pages and hence invisible. Write the jobs into $DFO_JOB_DIR/JOBS_NAVBAR:

trendPlotter -r <report> -N -f

and execute it say once or twice per month. Find the current operational JOBS_NAVBAR for FLAMES/GIRAFFE here.

Note: the trendPlotter output pages have a meta tag in their header to auto-refresh every 60 seconds, and to be not cached. Also, the score png files, as well as the main png and the closeup png files are linked with a unique ID (generated at runtime from the current date and the PID). Together, these tricks provide an always up-to-date browser display.

Parallel execution

You can run multiple trendPlotter instances at the same time. Each instance has its own $JTMP_DIR created as $TMP_DIR/$$ and deleted after execution. If called with option -f, the result

This new feature has two main advantages:

• you need no longer care about trendPlotter processes running against each other. So far there was a safety mechanism which put the second instance on hold as long as an earlier instance was detected.
• you gain performance, roughly by a factor of 2 if your system has 2 cores.

A more detailed performance analysis shows that one can actually gain up to a factor of 3 (Figure 1), but then the system is heavily loaded (up to 20 parallel jobs in an experiment). In Figure 2 the system load is shown to increase steeply beyond a number of 6 or so parallel jobs.

The obvious way of gaining performance is to transform the jobs file JOBS_TREND to launch up to 6 jobs in parallel. An operational example is available under http://www.eso.org/~qc/dfos/reference/JOBS_TREND. This jobs file executes almost 3 times as fast as a linear sequence (!), and still permits interactive activities in parallel.

	Figure 1. Total execution time vs. number of parallel trendPlotter jobs.
	Figure 2. System load (measured by unix 'uptime') vs. number of parallel trendPlotter jobs. Up to 6 jobs can be run with 'reasonable' load values (around 2).

Test details: all numbers measured on the(now de-commissioned) host dfo23; no other processes running; all tests done with GIRAFFE BIAS report (5 plots, 10 data sets). Each job shifted by 1 sec against precursor.