WFC3 Data Handbook

8.2	Evaluating the Amount of Persistence in Science Images

As described above, persistence in an image depends on images that have been obtained previously. However, when an observer retrieves his/her data, the observations that came before will most likely be proprietary and unaccessible by the observer. To address this difficulty, the WFC3 team has developed a set of prototype software that estimates the persistence in an observers image. Although not currently part of the standard calibration pipeline, the WFC3 team runs these persistence estimation tools on all WFC3 IR images. The tools produce an html file summarizing the amount of persistence in the image as well a several fits files that can be used in downstream analysis of the HST data.

The output products, which consist of a combination of html files and a number of fits files, can be retrieved by observes through MAST using a special screen located at:

http://archive.stsci.edu/prepds/persist/search.php.

The query screen is shown in Figure 8.3. Generally, these output data products are available a few weeks after an IR observation has taken place.

There are two types of persistence that generally affect data analysis differently. We define external persistence as persistence that is generated by an earlier visit, and internal persistence as persistence generate within the same visit as the image. External persistence is not within the control of the observer, and can appear anywhere in an IR image. Internal persistence is usually less important, since at least in the case of small dithers, it affects regions of the detector which are in the wings of the point spread function of bright sources. The results of a query from the persistence query screen for Visit H of program ID 11189 are shown in Figure 8.4. The columns EXT1, EXT2 and EXT3 indicate that 0.04%, 1.22% and 5% of the pixels were predicted to have external persistence exceeding 0.1, 0.03, and 0.01 e-/sec. The columns labeled TOT1, TOT2, and TOT3 provide the percentages for the sum of external and internal persistence. If all of the estimates of persistence for a dataset in the results screen, particularly those having to do with external persistence, are zero then one should not need to investigate further.

Observers should check for persistence as part of their evaluation of the science data quality of their observations. Observers should do so early enough that they have an opportunity to submit a HOPR if the data are so-compromised by the existence of external persistence in their images that the science they proposed cannot be carried out. Data analyzers should also check to determine whether they need to take persistence into account in analysis.

If the summary estimates in the results screen indicate there is persistence, then more information about the amount of persistence and its location in the images can be obtained by clicking on the links associated with data set column in the results screen. (If the data are still proprietary, you will be asked for your MAST login name and password at this point). This will bring up a page that contains additional text as well as a number of images showing where persistence is predicted. There are also various thumbnail images, which are supposed to show how well the persistence is removed by the model in selected areas of an image. The utility of these thumbnails varies, depending upon the crowding of the image with real sources. An algorithm is used to try to select regions of the image which have the largest amounts of persistence but which are not very crowded, so that it is easy to establish the sky level and automatically generate thumbnails. But this algorithm is not very sophisticated, and in many cases, all of the regions with significant persistence lie in regions of the image that contain brighter *real* objects. Nevertheless, inspecting the images is recommended, and should help one to determine, qualitatively, whether persistence is likely to affect the science one is attempting to carry out.

Figure 8.4: MAST search screen used to retrieve information about persistence

The software applied to all WFC3/IR images also produces several fits files, as indicated in Table 8.2, for each flt file in a dataset. The *_extper.fits and *_persist.fits files contain predictions of the external and total persistence in a dataset. The *_flt_cor.fits file is an flt file with the total persistence subtracted. These files can be retrieved from MAST as a tar.gz file by clicking one of the Visit links; each Visit must be retrieved separately. In situations where there is some persistence predicted in a science image, displaying these fits files along with original data in DS9 or some other image display is ultimately the best way to determine whether persistence needs to be taken into account in down-stream analysis of the data.

Table 8.2: Extra Files Associated with Persistence Correction


Generic Filename	Contents
_extper.fits	Prediction of external persistence
_persist.fits	Prediction of total persistence
_flt_cor.fits	flt file with persistence model subtracted