Files names and extensions used for ACS (J)

The file suffixes given to ACS data products are described in the table below and closely mimic the suffixes used by STIS. The initial input files to the calibration pipeline are the raw (RAW) files from Generic Conversion and the association (ASN) table, if applicable, for the complete observation set.

 

For CCD images, a temporary file, with the suffix BLV_TMP, is created once bias levels are subtracted and the overscan regions are trimmed. This file is renamed with the FLT extension after the standard calibrations (flat fielding, dark subtraction, etc.) are complete. The FLT files will serve as input for cosmic ray rejection (if required). For CR-SPLIT exposures, a temporary 'cr-combined image' (CRJ_TMP) is created and then renamed with the CRJ extension once basic calibrations are complete. Single MAMA images are given the FLT suffix once calibrations are complete. By definition, these images do not have an overscan region and are not affected by cosmic rays. The calibrated products of a REPEAT-OBS association will be several individually calibrated FLT exposures and a summed flat-fielded (but not cosmic-ray cleaned) SFL image.

 

Names Suffix Description Units Example
jxxxxxxxx_raw _RAW Raw data DN J8C032XUQ_RAW
jxxxxxxxx_asn _ASN Association file for observation set   J8C032050_ASN
jxxxxxxxx_spt _SPT Telemetry and engineering data   J8C032XUQ_SPT
jxxxxxxxx_trl _TRL Trailer File with processing comments   J8C032XUQ_TRL
jxxxxxxxx_flt _FLT Calibrated, Flat fielded individual exposure electrons J8C032XUQ_FLT
jxxxxxxxx_crj _CRJ Calibrated, CR-rejected, Combined image electrons J8C032XUQ_CRJ
jxxxxxxxx_sfl _SFL Calibrated, Repeat-Obs, Combined image electrons J8C032XUQ_SFL
jxxxxxxxx_drz _DRZ Calibrated, Geometrically Corrected, Dither-Combined image electrons/sec J8C032050_DRZ

Association Tables

Association tables are useful for keeping track of the complex set of relationships that can exist between exposures taken with ACS, especially with REPEAT-OBS, CR-SPLIT, and dithered exposures. Images taken at a given dither position may be additionally CR-SPLIT into multiple exposures. In these cases, associations are built to describe how each exposure relates to the desired final product. As a result, ACS associations will create one or more science products from the input exposures, unlike NICMOS or STIS associations. The relationships defined in the association table determine how far through the calibration pipeline the exposures are processed and when the calibrated exposures are combined into sub-products for further calibration. ACS data files are given the following definitions:

  • An exposure is a single image, the "atomic unit" of HST data.
  • A dataset is a collection of files having a common root name (first 9 characters).
  • A sub-product is a dataset created by combining a subset of the exposures in an association.
  • A product is a dataset created by combining sub-products of an association.

ACS association tables were designed to closely resemble the NICMOS association format, with three primary columns: MEMNAME, MEMTYPE, and MEMPRSNT. The column MEMNAME gives the name of each exposure making up the association and output product name(s). The column MEMTYPE specifies the role the file has in the association. A unique set of MEMTYPES specific to ACS were adopted to provide the support for multiple products. 

Exposure types in ACS associations. The suffix "n" is appended to the MEMTYPE when multiple sets are present within a single association.

MEMTYPE Description
EXP-CRJ Input CR-SPLIT exposure (single set)
EXP-CRn Input CR-SPLIT exposure for CR-combined image n (multiple sets)
PROD-CRJ CR-combined output product (single set)
PROD-CRn CR-combined output product n (multiple sets)
EXP-RPT Input REPEAT-OBS exposure (single set)
EXP-RPn Input REPEAT-OBS exposure for repeated image n (multiple sets)
PROD-RPT REPEAT-OBS combined output product (single set)
PROD-RPn REPEAT-OBS combined output product n (multiple sets)
EXP-DTH Input dither exposure
PROD-DTH Dither-combined output product

A sample association table for a two-position dithered observation with CR-SPLIT=2 is presented in the table below. This example shows how both MEMNAME and MEMTYPE are used to associate input and output products. The MEMTYPE for each component of the first CR-SPLIT exposure, JxxxxxECM and JxxxxxEGM, are given the type EXP-CR1. The sub-product Jxxxxx011 is designated in the table with a MEMTYPE of PROD-CR1. The last digit of the product filename corresponds to the output product number in the MEMTYPE. A designation of zero for the last digit in the filename is reserved for the dither-combined product.

The column MEMPRSNT indicates whether a given file already exists. For example, if cosmic ray rejection has not yet been performed by CALACS, the PROD-CRn files will have a MEMPRSNT value of "no". The sample association table below shows the values of MEMPRSNT prior to CALACS processing. 

Sample Association Table Jxxxxx010_ASN

MENNAME MENTYPE MENPRSNT
JxxxxxECM EXP-CR1 yes
JxxxxxEGM EXP-CR1 yes
Jxxxxx011 PROD-CR1 no
JxxxxxEMM EXP-CR2 yes
JxxxxxEOM EXP-CR2 yes
Jxxxxx012 PROD-CR2 no
Jxxxxx010 PROD-DTH no

Trailer Files

Each task in the CALACS package creates messages during processing which describe the progress of the calibration and which are sent to STDOUT. In the pipeline processing for other HST instruments, trailer files were created by simply redirecting the STDOUT to a file. Because multiple output files can be processed in a single run of CALACS, creating trailer files presents a unique challenge. Each task within the CALACS package must decide which trailer file should be appended with comments and automatically open, populate, and close each trailer file.

CALACS will always overwrite information in trailer files from previous runs of CALACS while preserving any comments generated by Generic Conversion. This ensures that the trailer files accurately reflect the most recent processing performed. The string CALACSBEG will mark the first comment added to the trailer file. If a trailer file already exists, CALACS will search for this string to determine where to append processing comments. If it is not found, the string will be written to the end of the file and all comments will follow. Thus any comments from previous processing are overwritten and only the most current calibrations are recorded.

As each image is processed, an accompanying trailer file with the '.trl' ending will be created. Further processing will concatenate all trailer files associated with the output product into a single file. Additional messages will then be appended to this concatenated file. Thus, some information is duplicated across multiple trailer files, but for any product processed within the pipeline, the trailer file is ensured to contain processing comments from each input file.

Linking trailer files together can result in multiple occurrences of the CALACSBEG string. Only the first, however, determines where CALACS will begin overwriting comments if an observation is reprocessed.

ACS Filestructure

The ACS calibration pipeline assembles data received from HST into datasets, applies the standard calibrations, and stores the uncalibrated datasets in the HST Data Archive. The structure of these data products is based on the STIS and NICMOS file format and consists of multi-extension FITS files which store science (SCI), data quality (DQ) and error (ERR) arrays.

ACS WFC data come from two CCD chips and are treated as separate observations, with a SCI, DQ and ERR array for each chip, and with both chips being stored in the same FITS file. The result is a FITS file with 6 data extensions plus a global header for a single WFC exposure. The WFC apertures are plotted with respect to the V2/V3 reference frame in Figure 2.2 in the Data Handbook and are oriented such that the x-axis is approximately to the right and the y-axis is approximately straight up. For pipeline data products, chip 2 is below chip 1 in y-pixel coordinates and was therefore defined as extension 1. Thus, the chip/extension notation is counterintuitive. To display the science image for chip 1, the user must specify the extension 'file.fits[sci,2]'. Similarly, the data quality and error array for chip 1 are specified as [dq,2] and [err,2].

A single HRC or SBC exposure comes from a single chip and has only 3 data extensions plus a global header. While the raw and calibrated WFC images contain 6 data extensions, the drizzled product will always contain 3 data extensions, no matter which detector was used. When WFC data is drizzled, both chips are included in a single FITS extension.

Figure 2.1 illustrates the basic format for storing ACS images. An uncalibrated (RAW) exposure contains a primary header plus, for each chip, a SCI extension (in 16-bit integer format), an empty ERR extension, and an empty DQ array. The calibrated product from CALACS contains a primary header plus, for each chip, a SCI extension (in 32-bit float format), an ERR extension (32-bit float), and a DQ extension (16-bit integer).

MultiDrizzle removes the effects of the geometric distortion and produces a drizzled (DRZ) product which has the multi-extension FITS file shown in Figure 2.1. The SCI extension contains the distortion-corrected data (32-bit float), the WHT extension contains the weight mask (32-bit float), and the CTX extension contains the context image (32-bit integer).