HIFI Instrument and Calibration page for Archive Users

 

 

Welcome to the HIFI instrument and calibration page. You will find here a set of top-level information about the HIFI instrument and its characteristics, together with recommendations on how to best exploit the archival data offered for this instrument in the Herschel Science Archive (HSA). Because more detailed and targeted documentation is often available somewhere else, this page is essentially a gateway to all relevant information needed to best use the HIFI products.

HIFI Documentation

A quite extensive set of documentation is compiled in the HIFI section of the Herschel Explanatory Legacy Library (HELL). In order to get acquainted with the HIFI products, and how to best exploit them, the following top-level documents from the HELL should be read first:

  • If you want to know whether HIFI is the right instrument for your science, please check the HIFI Quick-Start Guide.
  • If you want to use HIFI data, and need to become familiar with their characteristics and calibration, you should read the relevant sections of the HIFI Handbook.
  • In order to be aware of the different HIFI product flavours available in the HSA, please read the HIFI Products Explained, as well as the corresponding section below.
  • In case you want to analyse your HIFI data within the Herschel Data Processing Environment (HIPE), please read the HIFI Data Reduction Guide, as well as the corresponding section below.

HIFI Data Products

All HIFI products are publicly accessible through the Herschel Science Archive (HSA). An overview of the available products is given in the HIFI Products Explained document, and can be summarised as follows:

  • Products generated by the standard instrument pipelines: these have been processed essentially by HIPE 14.1 (see also Data Processing section below), and cover both observations performed during the Routine Science phase, and during the early Performance Validation and Routine Calibration campaigns. For the latter products, special care needs to be taken in case of observations obtained in a non-standard fashion - recommendations are given in this document.
  • User-Provided Data Products (UPDPs): these are products delivered by science consortia to the Herschel Science Centre. All UPDPs stored in the HSA can also be checked and downloaded from the following page:
  • Highly-Processed Data Products (HPDPs): these are products delivered by experts from the Instrument Control Centres and the (NASA) Herschel Science Centre. They offer essentially quality-enhanced (improved calibration) and/or complementary products to those processed with the standard pipelines. All HPDPs stored in the HSA can also be checked and downloaded from the following page:
  • Ancillary Data Products: these are products not necessarily linked to any observations, sometimes of lesser relevance to the scientific end user. They, however, provide important information concerning the instrument spatial responses, or more engineer-oriented data pre-launch and during the mission. All Ancillary Data Products stored in the HSA can also be checked and downloaded from the following page:

The vast majority of the HIFI products can be considered science-ready. The main residual artefacts from the standard calibration pipeline are summarised in the following section, while recommendations on how to deal with them are given in the Data Processing section below.

HIFI Performance and Calibration

HIFI used similar calibration concepts to those of other heterodyne radiometers operated in the millimetre and sub-millimetre domains. The precepts applied to its calibration can be divided into three main components: intensity, frequency, and spatial calibration. More details on how they were implemented on this instrument and the performance they allowed to achieve can be found in the following top-level documentation:

The calibration parameters used in the processing pipeline are stored in the so-called calibration tree. This set of files is present in any observation context downloaded from the HSA, but it can also be retrieved in the legacy repository area, together with a list of updates applying to each version.

The output of some of the calibration characterisation effort has also been compiled as Ancillary Data Products (e.g. sensitivity spectra as a function of tuning frequency), and can be found at the corresponding repository.

We provide, in the following, the basics behind each of the calibration components, with links to the most relevant documentation:

Intensity Calibration

The intensity calibration of HIFI is based on a three-point measurement towards internal black body sources and a blank sky ("OFF") position, bootstrapped to a surface model of the sub-mm emission of Mars in order to account for the imperfect optical coupling to the target of interest. The calibration framework developed for HIFI is described in the following document:

This calibration framework aimed at cancelling out all instrumental responses involved in the data taking. On some occassions, however, residual artefacts are still present in the pipelined data, essentially in the form of spectral baseline distorsion (standing waves, etc) or spurious signals. Section 5.3 of the HIFI Handbook gives further details on how they manifest in the HIFI data. In the Data Processing section, we give some information on how to deal with them.

The intensity calibration uncertainty of HIFI is summarised in the following table. It is further detailed in Section 5.8 of the HIFI Handbook.

  Band
  1 2 3 4 5 6 7
Absolute line and continuum flux accuracy (%)
Total Uncertainty on the Ta* scale 1.6-4 1.3-3 1.5-3 2-3 3-4 5-5.5 6
Beam Efficiency 1.1 1.4 1.3 1.3 1 1.2 1.3
Aperture Efficiency 1.1 1.4 1.3 1.3 1.1 1.2 1.3
Planetray model 5 5 5 5 5 5 5
Flux calibration repeatability (%)
Line flux repeatability -- 3-4 5-8 -- 6-9 9-12 11-12
Continuum flux repeatability 2 3-3.6 2.5-5 6 6.5 8 10

 

Frequency Calibration

The frequency calibration of the two HIFI spectrometers (the Wideband Spectrometer - WBS, and the High Resolution Spectrometer - HRS) relied primarily on the on-board master oscillator. For the WBS, an extra step involved regular measurement of an instrument-generated reference harmonic spectrum in order to calibrate out short term drift in its spectral response. The calibration framework developed for HIFI is described in the following document:

The frequency calibration uncertainty of HIFI is summarised in the following table. It is further detailed in Section 5.7 of the HIFI Handbook.

  Resolution Bandwidth (kHz) Accuracy (kHz)
HRS (high-res. mode, Hann apodisation) 207-364 20
Wide-Band Spectrometer 1100 100

 

Spatial Calibration

HIFI used Mars as prime intensity calibrator. Based on a surface model of Mars, dedicated measurements were performed in order to derive the detailed HIFI Point Spread Function (PSF), or beam, as well as the telescope coupling efficiencies.

Information about the model can be found in the Herschel calibrator page: The HIFI beam and efficiencies are described in Sections 4.3 and 5.9 of the HIFI Handbook, but the information can be directly accessed via the following products and documents:

HIFI Data Processing

The final pipeline products served in the archive have been processed for their vast majority with the pipeline version 14.1. About a dozen of isolated observations required reprocessing with the following software version 14.2. The last version of the Herschel Interactive Processing Environment (HIPE) was version 15.0.1, but this version was optimised for interactive analysis, and no product regeneration was involved. Reprocessing products with HIPE 15.0.1 should, however, yield the same outcome as those stored in the HSA.

In case you wish to perform data reduction or analysis with HIPE, version 15.0.1 can be downloaded at the following page:

There are still a handful of known issues with HIPE, and those are compiled in the following page:

Users should also be warned that a fraction of the User-Provided Data Products (UPDPs) were generated with an earlier version of the data processing, something dating several years from now. Older products will hold caveats which have now been fixed in the latest archival data. In case you need to work with products generated with an older version of the software, please check the HIPE DP known issues page, as well as the list of specific caveats and shortcomings of previous versions of the pipeline.

The HIFI products that should be primarily considered for science are those nick-named as standalone browse products, which are in effect the Level 2.5 products (highest processing level for HIFI) generated by the pipeline. Of course, lower level products are also available in the HSA - see the HIFI Products Explained for further details. As indicated in the performance and calibration section, most of the HIFI products can be considered science-ready. However, residual artefacts can still exist in some data, and they need particular attention. The following workflow to make best use of the HIFI products can be considered (see also Section 5 of the HIFI Quick-Start Guide):

  • Data inspection: you should pay attention to the "Quality Report" provided in the corresponding panel of the HSA query result table. On top of that, check the overall quality of your baseline to decide whether further interactive processing is needed (see next bullets). Finally, check whether alternative/better products might exist in the form of Highly-Processed Data Products (HPDP).
  • On rare occasion, re-processing of the data from lower level products might be beneficial. Section 6.4.1 of the HIFI Handbook describes the cases that might be eligible for this. Note that this can only be done within HIPE.
  • When baseline quality is not sufficient for immediate exploitation of the HIFI spectra, post-processing is necessary. This will mostly take the form of baseline cleaning (e.g. standing wave removal, or spurious feature masking). Section 6.4.2 of the HIFI Handbook describes the various use cases and recipes applicable to this situation. Note that this is not necessarily done in HIPE, although dedicated tools have been developed there for that purpose.
  • When you are happy with your products, you might want to export them for further use in other data analysis software. A particular note to users wishing to use Gildas/CLASS is that Class can now directly read HIFI FITS files generated by the pipeline and served by the HSA. More details can be found in this technical note: Importing Herschel-FITS into CLASS (Bardeaux et al. 2015).