SAS Thread - xmmextractor - XMM-Newton

Analysis chain for point-like sources (xmmextractor)

Description

Starting from the uncalibrated files stored in the Observation Data File (ODF, see Short Guide to XMM-Newton Science Data Products), calibrated events files are produced (using most up-to-date calibration files) from which in turn spectra and light curves can be extracted using source-specific filter criteria. This is a multi-step procedure that needs to be carried out for each instrument separately.

The purpose of xmmextractor is to combine all these steps with a single command. All user input is provided via a configuration file. A default configuration file can be generated, based on the individual situation (e.g., pointing coordinates, exposure times, exposure numbers). A separate tool, odfParamCreator, is available for the creation of a default configuration file that can then be edited with a text editor.

Preparation

The ODF can be downloaded from the XMM-Newton Science Archive, XSA.

xmmextractor can be run from any working directory. All output will be stored in the current working directory. Various subdirectories will be created in which the products will be stored.

The following preparatory steps are required before running xmmextractor:

• Download ODF data from XSA
• Initialize HEADAS/HEASOFT
• Initialize SAS, following the SAS Startup Thread.
  The tasks cifbuild and odfingest do not need to be run unless a customized session is to be generated (see below).
• As part of the SAS setup, the environment variable SAS_ODF needs to point to the ODF directory (full path).
• Create and change to the directory where the output should be stored. This directory will be called analysis directory.

The current implementation of xmmextractor only works reliably without any parameters. A call without parameters produces a full extraction of all data from all instruments in all modes. For a description of a default run, the following steps can be skipped, and proceed to Usage.

Preparation of a configuration file

A customized xmmextractor session is foreseen but currently not reliably possible. The concept of customizing an xmmextractor run is still explained here.

The infrastructure for customizing an xmmextractor run allows the flexiblility to analyse data from only a single instrument, or producing only spectra or light curves. A customized configuration file can be created and later provided when calling xmmextractor. The configuration file has to be formatted in the Extensible Markup Language (XML) which follows a hierarchical structure. A default file can be generated for later editing using the SAS task,

odfParamCreator outputFileName=rundefault.xml

This requires the prior setup of the SAS including the running the tasks cifbuild and odfingest.

The resulting xml file (in this case rundefault.xml), can be edited with a text editor. The structure of the configuration file is described in the Package description of odfParamCreator. The default configuration file directs xmmextractor to produce events files, spectra, and light curves for all instruments. Non-default extractions such as extractions only for one instrument, can be communicated to xmmextractor by editing the configuration file accordingly. An example is given below,

The first block, <OBSERVATION>, contains performance setup in the format: <PARAM id="analysisoption" default="0:all"/>, where the parameter name is given in quotes behind id= and the parameter value in quotes behind default=. The first parameter, analysisoption, determines which level of products are to be extracted. The default, 0:all, indicates that the full analysis is performed. Other options allow extraction of only events files, only spectra or light curves, etc. All options are explained in the description of xmmextractor. The remaining parameters are self-explanatory. The data reduction can be turned off for each instrument by changing default="yes" to default="no" behind the corresponding instrument for which no data analysis is desired.

The remaining blocks contain configuration information for each available instrument and are named <INSTRUMENT value="...."/>, where the name of the instrument is given in quotes. Within each instrument block, a separate block is present for each exposure, where the mode, exposure id, exposure time, and a flag whether or not this exposure is to be processed are given. Replacing process="yes" to process="no" means that the exposure is not processed. For each exposure, the products that the tool can produce are listed in blocks within which the task setup can be modified. Again, each yes can be replaced by no if not wanted. Parameter names and values can be modified. Not all details can be given here but should be found in the respective documentation for each task.

Usage

The most simple use and only fully functional mode is to type on the command line,

  xmmextractor >& screenoutput.log

where all screen output will be piped into the log file screenoutput.log if the command and the name of the log file are separated by >&. A default configuration file will be generated and can later be used to re run the tool with exactly the same setup. A copy can be created that can be modified and used to run the tool again with the modified setup.

After the configuration file has been adjusted to the individual needs, the tool can be run, giving the name of the configuration file:

  xmmextractor paramfile=rundefault.xml >& screenoutput.log

Note that without events files, no other products can be produced. Thus, events files must either already exist in the right directories, or the tool has to be run with analysisoption=1:events before using other options. If EPIC events files are to be recycled they need to be stored in the respective subdirectories pn or mos under the working directory and must be named according to the SAS naming convention

• pn
  REVN_OBSID_EPN_SEEE_ImagingEvts.ds
  REVN_OBSID_EPN_SEEE_TimingEvts.ds
• mos
  REVN_OBSID_EMOS[1,2]_SEEE_ImagingEvts.ds
  REVN_OBSID_EMOS[1,2]_SEEE_TimingEvts.ds

with REVN the 4-digit revolution number, OBSID the 10-digit observation ID, and EEE the exposure number.

If the tool is run again, existing files will not be overwritten. In particular, log files will be appended, and if events files have already been produced (which can take a long time), they will be used for the extraction of new light curves and spectra, for example for other sources in the field. Pipeline products can be used, but have to be renamed to SAS naming convention.

If events files have to be created, the full task can run for some 20-30 minutes or more. While running, brief pop ups of ds9 windows will occur for the only purpose of creating image plots as screen shots of ds9. They can be ignored or closed by hand.

Products

All products will be stored in the analysis directory from where the tool was launched. Performance output can be inspected from the three files,

• xmmextractor.err: Containing errors
• xmmextractor.warn: Containing warnings
• xmmextractor.info: Containing runtime information

It is strongly recommended to follow up all errors and warnings before using the science products. If messages are not clear, the XMM-Newton HelpDesk can be contacted. The science products are stored in the subdirectories that have been generated in the analysis directory. In the following, the contents of each directory that was created is given: