Herschel Data Products - Herschel
Herschel Data Products
Overview of Herschel standard products
A wide range of Herschel data products are available, both in 'scope' and 'level'. Most of them are automatically generated by systematic pipeline processing performed at the Herschel Science Centre (HSC) using the Herschel Data Processing system and made available through the Herschel Science Archive (HSA). These products are labelled with the version of the SPG (Standard Product Generation) software used for their production. Others, interactively generated, may eventually become part of the HSA at a later stage of the mission.
The Herschel products (not all of direct interest for the observers) consist of:
- Observational products
- Contain the scientific data resulting from the Herschel observations
- Classified depending on the level of the processing of the data they contain, ranging from raw data to highly processed scientific data (see below)
- Generated per observation (AOR), in contrast with highly processed products that may result from the combination of data from several observations (AORs)
- Auxiliary products
- Contain all Herschel non-science spacecraft data required directly or indirectly in the processing and analysis of the scientific data.
- Normally generated per Herschel Operational Day, with the exception of the Herschel Pointing Product that is generated per observation
- Calibration products
- Contain the parameters that characterise the behaviour of the satellite and the instruments.
- There are uplink and downlink calibration products. Downlink calibration products are used in the processing of the raw data to produce astronomically calibrated products in which the instrument artefacts have been removed.
- Quality control products
- Gather a summary of the information required to evaluate the technical quality of the executed observation and the products generated, and provide a global quality assessment
- User Provided Data Products (UPDPs)
- Interactively reduced data provided to the Herschel Science Centre by the observers (initially from the Herschel Key Program consortia only, that commited to do that explicitly, but not excluding other programmes as well, on a voluntary basis)
- Stored in the HSA and made available to the astronomical community after validation
- Must follow provided guidelines in terms of format and associated documentation
- A compilation of the currently available User Provided Data Products is provided here
- Highly Processed Data Products (HPDPs)
- Enhanced products provided by experts in the ground segment and very soon stored in the HSA
- A compilation of the planned HPDPs that will eventually be ingested into the HSA is provided here
- Ancillary Data Products (ADPs)
- Data (products, tables, plots, etc..) generated in the course of the different phases of the Herschel mission which are not necessarily linked to a particular observation, but which contain valuable additional information like e.g. the planetary, asteroid and stellar models used by the various instruments for their calibration, PSFs, trend analysis plots, etc.
- A compilation of the planned ADPs that will eventually be provided through the HSA is provided here
Overview of Herschel Data Product levels
The various Herschel Data Product levels can be briefly described as follows:
- Level-0 data products
- Raw telemetry data as measured by the instrument. They might be minimally formatted before its ingestion into the Herschel Science Archive. They are automatically generated by the data processing pipeline
- Level-1 data products
- Detector readouts calibrated and converted to physical units, in principle instrument and observatory independent
- Level-2 data products
- Level-1 data further processed to such a level that scientific analysis can be performed
- For optimal results many of the processing steps involved in the generation of level-2 data may require human interaction
- Should be suitable for Virtual Observatory access
- HIFI Level-2.5 single-point data products are stitched (i.e. only one concatenates spectrometer sub-band) spectra for each of the polarisations and backends applicable to the observation
- HIFI Level-2.5 map data products are regridded cubes for each of the polarisations and backend sub-bands associated to a given observation. The cube dimension is derived from the geometry of the executed map. For moving targets, the maps are provided in the co-moving frame
- HIFI Level-2.5 spectral scan data products are deconvolved Level-2 spectra (from both WBS-H and WBS-V data) using default parameters.
- PACS Level-2.5 photometric products are maps (produced with JScanam, Unimap and the high-pass filter pipelines) combining scan and cross-scan AORs taken on the same sky field
- PACS Level-2.5 spectroscopic products combine two observations obtained on-target and on a nearby reference off-position in the unchopped range-scan observing mode
- SPIRE Level-2.5 products are created from combination of Level-1 detector timelines and observations including the following types:
- Pairs of scan maps on the same target position taken separately in the nominal and orthogonal directions
- Sets of parallel mode groups of scans (2+) over the same sky region at various orientations
- Groups of overlapping Large Map mode observations made in single or nominal+orthogonal scan direction
The lists of observations combined to generate the Level-2.5 PACS and SPIRE data products are given in each of the above links.
Level-3 data products are available for PACS and SPIRE
- PACS Level-3 photometric products are combinations of Level-2.5 JScanam and Unimap overlapping maps on a given field generated using the Mosaic task of HIPE.
- PACS Level-3 spectroscopic products are provided only for pointed chopNod observations taken in SED mode and they are combined spectrum tables derived from Level-2 products corresponding to several observations of the same target.
- SPIRE Level-3 products are mosaics obtained by merging all or a subset of contiguous observations (tracked SSO observations or fixed maps that are known to contain SSOs are excluded). This leads, in some cases, to very large maps which are broken up into reasonable smaller groupings. For instance, the Galactic Plane is cut into chunks of about 15 degrees length with some adjustments around the Galactic Centre.
The lists of observations combined to generate the Level-3 PACS and SPIRE data products are given in each of the above links.
- Standalone Browse Products
Since 17 March 2014 the new version of the Herschel Science Archive (HSA 5.2) offers the possibility to download 'Standalone Browse Products' from the Herschel User Interface (using the standard retrieval options or by right clicking on the postcard) and from the Postcard Gallery. 'Standalone Browse Products' are not necessarily 'science ready' products. They are provided for 'quick look' purposes. At this point, level 2 and level 2.5 FITS products generated by the standard Herschel data processing pipelines are distributed as 'Standalone Browse Products'. They will evolve in the future with the objective to become simpler and easier-to-use products.
HERSCHEL PRODUCT DEFINITION DOCUMENT
The Herschel Product Definition Document (PDF and HTML) gives further information on the definition of the different Herschel product types (Chapter 2 of the document), on the metadata keywords (Section 2.8), the naming convention used for the FITS files (Section 2.10 of the document), and provides detailed information on every instrument product (Chapter 3 for HIFI, 4 for PACS and 5 for SPIRE) and on the Auxiliary and Quality products (Chapter 6). Very detailed tables describing in detail the format of every Herschel product in the Herschel Product Definition Supplement (PDF and HTML).
Data proprietary rights
All observations made in the first year of the routine phase had proprietary times of 12 months, while for all observations made later, the proprietary time was 6 months, with a simple 'bridging scheme' so that no observation was made public before observations that were executed earlier became public as well. The proprietary time applies to each observation individually, counted from the day when the data was made available to the data owner. However, a scheme was put in place whereby the Herschel Project Scientist and the HOTAC Chair in consultation could grant additional proprietary time to certain large programmes, in order to prevent the release of improperly or inhomogeneously calibrated or processed data.