Документ взят из кэша поисковой машины. Адрес оригинального документа : http://www.atnf.csiro.au/research/workshops/2015/ASKAPEarlyScience/ASKAPSoftware_Oct2015.pdf Дата изменения: Wed Oct 7 08:06:12 2015 Дата индексирования: Mon Apr 11 17:46:28 2016 Кодировка:

Delivery of So5ware for ASKAP Early Science
Ma#hew Whi*ng
Credit: Alex Cherney/terrastro.com

CSIRO ASTRONOMY & SPACE SCIENCE


Expected capabili*es for Early Science
Program Cimager Ccalibrator Cbpcalibrator Cflag Ccalapply Ccontsubtract Linmos Cmodel Mssplit Selavy Purpose Imager (con9nuum MFS, spectral cubes) Gains and leakage calibra9on Bandpass calibra9on Flagging Apply calibra9ons solu9ons to measurement sets Con9nuum subtrac9on Linear mosaicking Model image generator Take a subset of a measurement set Source finding (con9nuum, spectral-
line, RM synthesis) Quality evalua9on (of a basic form) Pipeline scripts to 9e the above together CASDA archiving of key data products

Early Science SoQware Delivery -
Whi9ng | ASKAP Early Science Workshop | October 8, 2015


What will not be available
Early science processing will not have the full set of pipeline capabili9es expected for ASKAP Elements not available for early science include: · Transient imaging (5sec cadence) · Global sky model · Zoom mode imaging · High-
resolu9on (6km-
baseline) imaging · "Postage-
stamp" imaging

Early Science SoQware Delivery -
Whi9ng | ASKAP Early Science Workshop | October 8, 2015


Nature of processing pipeline
A "pipeline" is a set of scripts that 9e together individual ASKAPsoQ applica9ons Handles dependencies between the different jobs, and provides monitoring tools for progress and performance Controllable through a (rela9vely) small number of parameters Different modes available according to desired data products A prototype for this pipeline is currently available within the ACES team for tes9ng with BETA Not the final ASKAP pipeline ­ s9ll user-
driven

Early Science SoQware Delivery -
Whi9ng | ASKAP Early Science Workshop | October 8, 2015


Key data products
Calibrated visibili9es
Measurement set format

Con9nuum images (MFS)
+ PSF, Residual, Sensi9vity, Clean model

Spectral cubes
+ PSF, Sensi9vity, Residual, Clean model

Con9nuum cubes (full Stokes) Catalogues
Source finding for con9nuum, spectral-
line, polarisa9on Compliant with CASDA requirements

Early Science SoQware Delivery -
Whi9ng | ASKAP Early Science Workshop | October 8, 2015


Indica*ve data sizes
Visibility set 16416 channels, 5 sec 16416 channels, 30sec 304 channels, 5 sec 304 channels, 30 sec 1 hour 1004.7 167.5 27.9 4.6 8 hour 8037.7 1339.6 223.2 37.2 12 hour 12056.5 2009.4 334.8 55.8 (sizes in GB) Image product Con9nuum image Con9nuum cube Spectral-
line cube Pixel size 4K x 4K 4K x 4K x 304 4K x 4K x 16416 Disk size 64 MB 19 GB 1026 GB

Early Science SoQware Delivery -
Whi9ng | ASKAP Early Science Workshop | October 8, 2015


ASKAP Science Data Processor PlaQorm
· The galaxy system at Pawsey · 472 x Cray XC30 Compute Nodes
­ 200 TFlop/s Peak

· Cray Aries (Dragonfly topology) · Cray Sonexion Lustre Storage

­ 1.4 PB usable ­ 480 x 4TB Disk Drives, RAID 6 + Hot Spares ­ Peak I/O performance: 30 GByte/s

Early Science SoQware Delivery -
Whi9ng | ASKAP Early Science Workshop | October 8, 2015


Opera*on of Science Pipelines
Ingest pipeline runs automa9cally, wri9ng MS to disk, no9fying responsible team members Processing run by ACES (including secondees from science teams) Galaxy access not open to all science team members Compu9ng team provides support for plamorm & askapsoQ Other packages available, but liole support Data will be available for long enough to enable processing Need not be processed immediately Data cura9on will be necessary, however Post-
processing will not be done on galaxy ­ addi9onal applica9on for magnus
Early Science SoQware Delivery -
Whi9ng | ASKAP Early Science Workshop | October 8, 2015


Limits to pipeline processing on galaxy
There will be limits to galaxy availability for early-
science processing Other tasks will be on-
going:
· Processing of commissioning observa9ons · Integra9on/tes9ng/verifica9on of new soQware capabili9es & pipelines

Es9mate 30-
50% of galaxy system will be available for early-
science processing Crude es9mate of required 9me 500,000 node-
hrs for single processing of all proposed observa9ons

Early Science SoQware Delivery -
Whi9ng | ASKAP Early Science Workshop | October 8, 2015


Challenges
· Re-
engineering and scaling of imager framework · Compe9ng uses of galaxy · Pawsey & network infrastructure
· Ingest pipeline moving to Pawsey

· Calibra9on methodology
· Per-
beam calibra9on? · Polarisa9on calibra9on & 9mescales

· Tes9ng & valida9on of pipeline processing
· Engagement through ACES!



Early Science SoQware Delivery -
Whi9ng | ASKAP Early Science Workshop | October 8, 2015


ASKAP Compu*ng Plan: Milestones leading to Early Science
ID Descrip*on External Dependencies Target Status CMPT-
3 Delay/phase tracking, support for ADE-
6, 36 beams 48 MHz, ingest running at MRO ADE-
6 installed & verified 9 Nov 15

CMPT-
4 Ingest pipeline running at 40 G link from MRO to Pawsey to support ADE-
6, Pawsey available in Oct 36 beams, 304 MHz CMPT-
5 Support of basic early science capabili9es to start Array Release 2 (ADE-
12) commissioning >ADE-
10 installed & verified

End Nov 15 Apr 16

CMPT-
6 TOS and ASKAPsoQ ready At least 8 weeks of ADE-
to start Early Science 12 commissioning

Jun 16

Early Science SoQware Delivery -
Whi9ng | ASKAP Early Science Workshop | October 8, 2015


We acknowledge the Wajarri Yamatji people as the tradi;onal owners of the Observatory site.

Thank you
CSIRO Astronomy & Space Science Maohew Whi9ng Lead, ASKAP Science Data Processing t +61 2 9372 4683 E maohew.whi9ng@csiro.au w www.atnf.csiro.au/projects/askap
CSIRO ASTRONOMY & SPACE SCIENCE