Документ взят из кэша поисковой машины. Адрес оригинального документа : http://www.naic.edu/~palfa/doc/asp_processing.txt.old Дата изменения: Thu Aug 12 23:55:01 2004 Дата индексирования: Sun Apr 10 18:06:49 2016 Кодировка: Поисковые слова: http www.astronomy.ru forum index.php topic 4644.0.html

How to start and monitor the online PALFA data reduction on the ASP.

Notes by DRL/DJC, 12 August 2004.

Overview
--------

There are two main processing schemes currently implemented -- the
so-called "quick" option which ftps, demultiplexes and searches the
resulting individual pixels at low resolution (32 x 3.125 MHz channels
and 1.024 ms sampling); and the "long" option which currently searches
data at higher resolution (64 x 1.5625 MHz channels and 128 us sampling).
The "quick" processing must be run first and is designed to work while
the data are being collected on the WAPPs. The "long" processing can
only be run once the "quick" processing is complete and assumes that
the demultiplexed data are on the WAPP. Currently a standard FFT (seek)
and single-pulse searches (Cornell code) are run on both pipelines.
The results are written to /home/alfa/realtime/results on the ASP but
are best viewed through the php web server (see below).

NodeStatus
----------

SSH into aspmaster as user alfa. Paulo or any of the ASPers will tell you
the password.

Type "realtime" and a monitor/control GUI will pop up. Check that the
current state of all the processors is "stopped on request" or ends in
"done". It is worth checking that the processor usage is close to 0%, this
is shown for each node on the right side of the GUI. If this is not the
case, there is probably something still running and you should hit the
"Stop" button.

Before starting the processing, make sure the local disks on the nodes are
clean by hitting the "Clear Files" button. Check the disk space on each
node. If the number is highlighted in yellow then there is not enough
space and this node will not be used. If it is highlighted in green then
there may not be enough space for the whole run and may be not be used later.

You are now ready to start processing. Hit the "Start" button and you
should see the four ftp processes shown at the top change to blue and say
"waiting for data". The colours of all the nodes should turn blue and end
with "done". Once your first scan is finished, they will start to copy data
onto the local disks. Processing will begin once files are completely
ftp'd over (it takes of order the integration time to ftp the files).

Below the disk space and cpu monitor of each node there is a summary of
the files waiting to be processed. It is in the format: files in pool -
split files (incoming files). If the files in the pool becomes > 5 it will
highlight yellow. This may occur when a process is unable to finish
properly, check that the processor usage is at 0.0%. It can usually be
fixed by 'echo "none done" > /palfa/node?/status/status.proc?'.

For example, to manually clear the two processors "a" and "b" on
node 5, you would do the following:

echo "none done" > /palfa/node5/status/status.proca
echo "none done" > /palfa/node5/status/status.procb

As each file is successfully copied onto each node it will automatically
begin processing. You can look at the results by opening another window on
aspmaster and running the php web server by hitting the "View results"
button (see below for further details about viewing the output).

After the observing has finished the processing will need time to catch
up. When all the ftps and nodes turn blue the files should read 0 - 0 (0)
and the processor usage should drop to 0%. At this point you can click
"Stop" and in ~1 min all processes should turn yellow and report "stopped
on request".

If you wish to do the long processing you can click "Reprocess" once you
have finished the quick processing. The split files are copied to the pool
and processing occurs as before. Processing is ended in the same way as
with the quick processing.

If you wish to kill all processes immediately use the "Kill" button. This
may take several 10s of seconds to run since it has to ssh into each node
and kill any relevant processes. Once this is complete, you should click
on "stop" and then either "start" or "reprocess".

Notes: The CPU usage lags the GUI by ~40 secs
If you quit the GUI processing continues
After "stop" has been clicked some processes may continue for
several mins (up to 10 mins for the reprocess) so the CPU monitors provide
a good check. When these processes finish they will report "done"
overwriting the "stopped on request"


Candidate Viewer
----------------

The candidate viewer can be run from any browser on any computer on the AO
network, not just ASP.

Enter the URL http://aspmaster.naic.edu:8080/alfa/viewer.php and it will
show you the results for today.

At the top you can choose to see results for a specific MJD and scan
number range. This is useful as the more pointings that are displayed the
longer the page takes to build. You can also choose to view the quick or
long processing.

There is a box where you can enter your name or initials. This will allow
you to keep track of the beams which you have viewed.

NOTE: You need to click "update" after you make a change in the Min/Max
scan number or name boxes.

The pointings are listed below. You can view the candidates of each beam
by following the link. You can jump to the ps files of the top 3
candidates and single pulse search by following by the links below. Any
beam which you have viewed will be subsequently highlighted in green.

In the 'view candidates' page the top 10 candidates and the result of the
single pulse search are listed. Clicking on the figure will link you to
the full ps file. If there is a nearby pulsar (within 10 arcmin of the
central beam) in the pulsar catalogue it will appear at the top of the page.

NOTE: The position offset claimed for a known pulsar is from beam 0
at the moment. Plans are afoot to correct this position in future.

At the bottom of the page there are several links. "list" returns you to
the pointings list. "Next / Prev Beam" will take you to the next / prev
beam and will wrap to the next / prev pointing if necessary. "Next / Prev
unviewed beam" will take you to next / prev beam which your username
hasn't viewed, i.e. does not have a green box.

NOTE: If you go "back" rather than using the "list" link the green boxes
may not appear to be updated. Click reload/refresh to update.

If you use the same username from a different browser or at a different
time the beams which you have viewed will be retained.

NOTE: Wapp2 is the slowest to transfer so beams 2 & 3 are often the last
to arrive.
NOTE: creation of the PNG files is more robust than creation of the PS
files so occasionally links to the ps files may be missing,