All Hubble Space Telescope (HST) data files are stored in the Hubble Data Archive (HDA), which forms part of the Multimission Archive at STScI (MAST)
1.
HST Guaranteed Time Observers (GTOs), Guest Observers (GOs), and Archival Researchers can retrieve data in one of the following ways:
|
•
|
Via the use of external hard drives loaned by STScI: Users requesting large amounts of data (>100GB) over a short period of time may request the data be written to hard disk. The disk will be delivered to the user for reading, and then the disk must be returned. Use of this retrieval method must be coordinated through the MAST Help Desk ( archive@stsci.edu). The number of external hard drives is limited, so users are strongly encouraged to make arrangements in advance. For reference, a DVD can hold 4.7 GB.
|
All datasets retrieved from the HDA, regardless of the method used, will be in FITS (Flexible Image Transport System) format. Further information on
HST file formats is presented in
Chapter 2.
Non-proprietary data in the HDA can be retrieved electronically either by registered HDA users or via anonymous login. Proprietary data may not be retrieved except by a registered HDA user who has the permission of the program’s Principal Investigator (PI). Note that
HST PIs are
not automatically registered. PIs should register
before their first observations have been taken. All calibration observations as well as observations made as part of the GO Parallel programs are immediately public. All observations made as part a Treasury Program will either be immediately public or have only a brief proprietary period. The High-Level Science Products (HLSP) section of MAST also contains several sets of publicly available and fully reduced
HST data such as the Ultra Deep Field and the GEMS survey data. See
http://archive.stsci.edu/hlsp/index.html for a complete listing.
|
|
Advanced Camera for Surveys (ACS), Cosmic Origins Spectrograph (COS), and Wide Field Camera 3 (WFC3) users should note that the preferred option for data retrieval is from the HDA staging disk, via ftp/sftp. Users retrieving large numbers of ACS, COS, or WFC3 files should also consider requesting them on DVDs.
|
The HDA contains all observations ever made by HST and a catalog that describes the data. Each time a user makes a data request, the HDA delivers data which are either processed with the On-The-Fly-Reprocessing (OTFR) system or simply retrieved from a set of final calibrated and statically stored data, as listed in
Table 1.1.
Table 1.1:
HST Instrument Data Processing
The OTFR system reconstructs and calibrates the data at the time of retrieval. This allows users to obtain data calibrated with up-to-date reference files, parameters, and software. OTFR makes use of the OSS/PODPS Unified System (OPUS) system to orchestrate the processing of the data. OPUS retrieves the
HST telemetry data (POD files) from the Data Archiving and Distribution System (DADS) and creates raw data files in a step called Generic Conversion. Calibration reference files specific to the different modes of
HST operations are prepared and archived in the Calibration Data Base System (CDBS). During Generic Conversion, CDBS is queried to determine which reference files apply to the specific observation being processed. OPUS then calibrates the data using the specific instrument’s calibration software and reference files. After calibration, the files are returned to DADS for distribution to the user, as calibrated and/or raw files.
All STIS data collected before SM4 have been fully reprocessed and calibrated. They are now stored online, along with FOS, GHRS, and FOC data. This online data is available via FTP from archive.stsci.edu. (STIS data taken after SM4 will be processed via OTFR.) All WFPC2 data and NICMOS data taken before SM4 have been reprocessed and are stored statically in DADS.
Data for FOC, FOS, GHRS, HSP, and WF/PC-1 do not pass through OTFR or any pipeline. For FOC, FOS, and GHRS, final calibrated archives have been produced, since no further improvements in the calibration for these instruments are expected. The user is provided with a copy of the raw and final calibrated data from the static (final) archive once a request is made. For HSP and WF/PC-1, no reprocessing or recalibration has been done nor is any planned. Once raw data from these instruments are retrieved from the HDA, they need to be calibrated locally by the users.
Searches and retrievals are available through the MAST Web site, http://archive.stsci.edu. The MAST Web site allows cross correlation of HDA searches with other MAST mission archives. It offers simple previews of
HST datasets when available, as well as links to references citing a given dataset using the Astrophysics Data System (ADS). The MAST Web site is discussed in more detail in
Section 1.2.
The simplest way to register for HST data is to complete the form on the Web page at:
http://archive.stsci.edu/registration. If problems occur, registration requests may also be sent to the HDA Help Desk, at
archive@stsci.edu. The PI of each
HST proposal must request access (i.e., authorization) to their proprietary data for themselves, and for anyone else whom the PI wants to have access to them. PI retrieval permission is not granted automatically, for security reasons. PIs wishing to allow access to their proprietary data should make that request to
archive@stsci.edu. When registration is granted, your account will be activated automatically, and you will receive your username and password via e-mail.
The MAST Web site provides a wealth of useful information, including an online version of the
HST Archive Manual available at
http://archive.stsci.edu/hst/manual. Investigators expecting to work regularly with
HST and other datasets supported by MAST should also subscribe to the MAST electronic newsletter by sending an e-mail message to
archive_news-request@stsci.edu with the word “
subscribe” in the body of the message. Questions about the HDA can be directed to
archive@stsci.edu, or by phone to (410) 338-4547.
