Документ взят из кэша поисковой машины. Адрес
оригинального документа
: http://www.atnf.csiro.au/computing/software/askapsoft/sdp/docs/current/BETApipelines/pipelineUsage.html
Дата изменения: Unknown Дата индексирования: Tue Apr 12 13:03:46 2016 Кодировка: IBM-866 |
The pipeline scripts are now accessed through a specific module on galaxy - askappipeline. This is separate to the main askapsoft module, to allow more flexibility in updating the pipeline scripts. To use, simply run:
module load askappipeline
Note that for the linear mosaicking, if you want to get the beam centres based on the beam footprint specification (ie. using the ACES tool footprint.py), you will need to load the ACES module (to get the correct python packages) and have an up-to-date version of the ACES subversion repository. If you have not loaded the ACES module, it is likely the footprint.py task will fail, and mosaicking will be disabled.
Once loaded, the module will set an environment variable PIPELINEDIR, pointing to the directory containing the scripts. It also defines PIPELINE_VERSION to be the version number of the currently-used module.
To run the processing, you need to call processBETA.sh, providing it with a user input file that defines all necessary parameters. If you’ve loaded the pipelines module as detailed above, then this should be able to be run directly, like so:
processBETA.sh -c myInputs.sh
where the user input file myInputs.sh could look something like this:
#!/usr/bin/env bash
#
# Example user input file for BETA processing.
# Define variables here that will control the processing.
# Do not put spaces either side of the equals signs!
# control flags
SUBMIT_JOBS=true
DO_SELFCAL=false
# scheduling blocks for calibrator & data
SB_1934=507
SB_SCIENCE=514
# base names for MS and image data products
MS_BASE_SCIENCE=B1740_10hr.ms
IMAGE_BASE_CONT=i.b1740m517.cont
# beam location information, for mosaicking
BEAM_FOOTPRINT_NAME=diamond
FREQ_BAND=1
BEAM_FOOTPRINT_PA=0
# other imaging parameters
NUM_PIXELS_CONT=4096
NUM_TAYLOR_TERMS=2
CPUS_PER_CORE_CONT_IMAGING=15
This file should define enough environment variables for the scripts to run successfully. Mandatory ones, if you are starting from scratch, are the locations of either the SBs for the observations or the specific MSs.
When run, this file will be archived in the slurmOutputs directory (see below), marked with an appropriate timestamp so that you’ll be able to keep a record of exactly what you have run.
Important Note: The input file is a bash script, so formatting matters. Most importantly in this case, you can not have spaces either side of the equals sign when defining a variable.
The user is able to specify environment variables that directly relate to the parset parameters for the individual ASKAPsoft tasks. The input parameters are named differently, so that they are tied more obviously to specific tasks, and to distinguish between the the same parset parameter used for different jobs (eg. the preconditioning definition could differ for the continuum & spectral-line imaging).
The input environment variables are all given with upper-case names, with an underscore separating words (eg. VARIABLE_NAME). This will distinguish them from the parset parameters.
The following pages list the environment variables defined in defaultConfig.sh тАУ these can all be redefined in your input file to set and tweak the processing. The default value of the parameter (if it has one) is listed in the tables, and in many of the tables the parset parameter than the environment variable maps to is given.
Any measurement sets, images and tables that are created are put in an output directory specified in the input file (if not provided, they go in the directory in which processBETA.sh is run). There will be a file called PROCESSED_ON that holds the timestamp indicating when the script was run (this timestamp is used in various filenames). Also created are a number of subdirectories which hold various types of files. These are:
To provide the input data to the scripts, you can provide either the scheduling blocks (SBs) of the two observations, or provide specific measurement sets (MSs) for each case.
The measurement sets that will be created should be named in the configuration file. A wildcard %b should be used to represent the beam number in the resulting MSs, since the individual beams will be split into separate files.
Each step detailed below can be switched on or off, and those selected will run fine (provided any pre- requisites such as measurement sets or bandpass solutions etc are available). If you have already created an averaged science MS, you can re-use that with the MS_SCIENCE_AVERAGE parameter (see User Parameters - Preparation of the Science field data), again with the %b wildcard to represent the beam number.
Here is a summary of the workflow provided for by these scripts: