Combining OM fast mode time series into a single data set

Procedure

If you want to barycentrically correct the data, the process uses the SAS task, barycen, in which case you will need to set up the SAS following the guidance in the SAS start-up thread. In this case, you will also need (i) the Calibration Index file and (ii) the SAS summary file. You will need to have FTOOLS set up.

If you process the ODF yourself from scratch (e.g. using omfchain), the Calibration Index file will be the output of the cifbuild SAS task (by default, named ccf.cif), and the SAS Summary file will be created by the odfingest SAS task (by default, named *SUM*.SAS). If, instead, you are using pipeline processed products from the XSA, the Calibration Index file is a single file of the form P*CALIND*.FIT/FTZ, but you will still need to generate the SAS Summary file, *SUM*.SAS, using odfingest.

To use the barycen task, you will also need to assign the SAS_ODF, SAS_CCF and SAS_CCFPATH environment variables to (a) the SAS Summary file, (b) the Calibration Index file, and (c) the path to your repository of calibration CCF files, respectively. If you are not barycentrically correcting data, these 3 assignments are not needed.

You will also need the the OM Fast mode timeseries FITS files, which will have the form P*OM*TIMESR*.FIT/FTZ, whether taken from the pipeline products in the XSA or from self-generated products from omfchain.

Note that as some steps below involve overwriting of existing timeseries files, if you wish to keep the original timeseries files, you should make safe copies of them, though they can of course be regenerated or re-extracted from the archive tar file.

The steps are then as follows (the first 3 steps can be run in a loop, in a script, over all the timeseries files)

Step 1

If you do not want to barycentrically correct the data, skip to step 2. Otherwise, for each fast mode timeseries file, {file}, run barycen as,

barycen table={file}:RATE

It should not be necessary to provide the source coordinates on the command line as the task should find these itself, from header keywords in the timeseries files.

NOTE: The barycen task overwrites the TIME column of the input {file}.

Step 2

For each timeseries file (from step 1, if barycentrically correcting), add a column for the MJD (or BMJD if barycentrically corrected) using the FTOOL, fcalc as,

fcalc infile={file} outfile={file} clname=MJD expr="<mjdref>+(TIME/86400.0)" clobber=yes

where <mjdref> is the reference MJD extracted from the header of the FITS file. In fact, since <mjdref> is a fixed value of 50814.0, which can be confirmed using the FTOOL, fkeyprint, on any of the files, simply replace <mjdref> with 50814.0 in the above command. The output file is an overwrite of the input file, with the new column.

Step 3

For each timeseries file from step 2, add a column for the filter, again using fcalc,

fcalc infile={file} outfile={file} clname=FILTER expr="'{filt}'" tform="4A" clobber=yes

here {filt} is the filter name (e.g. UVW1), obtained from the header keyword, filter, for example, again using the FTOOL, fkeyprint. The output file is an overwrite of the input file, with the new column.

Step 4

Once step 3 has been run on all the timeseries files you want to combine, merge all the FITS files from step 3 using the FTOOL, fmerge. To do this, create a simple ascii file (<listfile>) containing a single column list of all the timeseries FITS files. Run the task as,

fmerge infiles=@<listfile> outfile=<finaloutputfile> columns='-'

At this point, you should have a single FITS output file (<finaloutputfile>) that can be analysed further or plotted. The MJD information (barycentrically corrected BMJD, if you performed step 1) will not be uniformly spaced, even if not barycentrically corrected, because, invariably, the start times of the separate input files will not conform to a uniform time grid. The data from each input file remain binned at the original binning of that fast mode exposure, in the output file. No further binning is applied, though the file can be binned subsequently, using other tools.

Using TOPCAT, STILTS, IDL, XRONOS or any other suitable/user-preferred plotting tools, one can then display the data, exploiting the filter information to, for example, select filter subsets and/or display the light curves of different filters in distinctive ways.

Steps 1-3 can be embedded in a script that loops over the files, with step 4 being executed after the loop.

A very basic C-shell script is provided that should enable a user to take a user-created ascii list of the fast mode time series to be combined, and generate a concatenated light curve (barycentrically corrected, if required) in one go*. To use the script, you will need to set up the HEASOFT/FTOOLS and SAS (SAS, only if barycentrically correcting the data) and ensure the PFILES environment variable is defined to point to the path to the FTOOLS pfiles. Some guidance is provided in the script header. The script must be made executable (chmod u+x <script file>).

The csh script is available here

*Please note that the XMM-Newton helpdesk can not provide any user support for troubleshooting, maintaining or updating the script.