SAS Thread - epic filterbackground - XMM-Newton

How to filter EPIC event lists for flaring particle background

Before proceeding with this thread, the following steps are mandatory.

• Follow the SAS Startup thread and define the environment variables SAS_ODF and SAS_CCF.
• Follow the How to reprocess ODFs to generate calibrated and concatenated EPIC event lists thread and produce EPIC-pn and EPIC-MOS event files.

Below you can find the step-by-step version of the thread or the GUI version which makes use of the user-friendly graphical interface to SAS, both producing the same end result. The procedure can be applied to both EPIC-pn and EPIC-MOS instruments, with some changes to the value of some parameters, which will be highlighted. For simplicity, to make the name convention valid for both instruments, file names use the word EPIC to refer to either EPIC-pn or EPIC-MOS indistinctively. With this naming convention it is assumed that EPIC.fits is the name of the unfiltered event file.

Procedure
 

• Step-by-step version

   

  1. Extract a single event (i.e. pattern zero only), high energy light curve, from the event file to identify intervals of flaring particle background:

      

       evselect table=EPIC.fits withrateset=Y rateset=rateEPIC.fits \
        maketimecolumn=Y timebinsize=100 makeratecolumn=Y \
        expression='Selection_Expression'
     
     where Selection_Expression is:
     
         #XMMEA_EM && (PI>10000) && (PATTERN==0) for EPIC-MOS
         #XMMEA_EP && (PI>10000&&PI<12000) && (PATTERN==0) for EPIC-pn
     
     Notice how the EPIC-pn energy range selected for producing the background light curve includes events only up to 12 keV. The reason for this is to avoid hot pixels being miss-identified as very high energy events. One can look at the produced light curve by running:
     
      dsplot table=rateEPIC.fits x=TIME y=RATE.ERROR

  2. Determine where the light curve is low and steady. Choose a threshold (count/second) just above the low steady background to define the "low background" intervals, to create the corresponding GTI file:
     
      tabgtigen table=rateEPIC.fits expression='RATE<=Rate_Expression' gtiset=EPICgti.fits
     
     where Rate_Expression is:
     
         RATE<=0.35 for EPIC-MOS
         RATE<=0.4 for EPIC-pn
     
     This value will be different for each data set (the value given here for each instrument represents a good reference value for a standard observation). Use this expression to filter the event list:
     
      evselect table=EPIC.fits withfilteredset=Y filteredset=EPICclean.fits \
        destruct=Y keepfilteroutput=T \
        expression='Selection_Expression'
     
     where Selection_Expression is:
     
         #XMMEA_EM && gti(EPICgti.fits,TIME) && (PI>150) for EPIC-MOS
         #XMMEA_EP && gti(EPICgti.fits,TIME) && (PI>150) for EPIC-pn
      
  3. The file EPICclean.fits contains the filtered EPIC event list and can be now used to produce scientific products. The file EPICgti.fits contains the definition of Good Time Intervals.

   

• GUI version

   

  1. Start xmmselect
     
      xmmselect table=EPIC.fits &
     
     First a window pops-up, asking if you wish to visualize the "[...] selection expression [...]" corresponding to "[...] data subspace information [...]". In practise, xmmselect is asking you if you wish to see the data screening expression, which was employed to generate the event list. The answer to this question does not affect the following steps.
     
     The xmmselect call pops-up a window as shown in Fig.1.
      
     
     Fig.1: The main xmmselect window
     
     In this window, we identify:
      
     • a data screening widget (top)
     • a data column panel (middle)
     • the buttons 1D region and 2D region, which allow to translate selection expression defined in a grace or ds9 window, respectively, into proper selectlib expressions
     • "action" buttons (bottom)
  
  
  2. Extract a single event (i.e. pattern zero only), high energy light curve, to identify intervals of flaring particle background. This is done by:
      
     • writing the following selection expression in the data screening widget: '#XMMEA_EM && (PI>10000) && (PATTERN==0)' for EPIC-MOS and '#XMMEA_EP && (PI>10000&&PI<12000) && (PATTERN==0)' for EPIC-pn
     • clicking on the radio button close to the TIME column in the data column panel
     • clicking on OGIP rate curve
     
     This will pop-up another window: the evselect parameter user interface (see Fig.2).
      
     
     
     Fig.2: The main evselect window. This particular GUI shows the case for EPIC-MOS.
      
     On this multi-panel window, one should at least:
      
     • click on the Lightcurve menu (and this leads you to the sub-panel shown in Fig.3)
     • define the output file name in the rateset widget (e.g.: rateMOS.fits)
     • define the time bin size (in seconds) in the timebinsize widget
     • click Run
      
     
     Fig.3: The Lightcurve panel in the evselect window. This particular GUI shows the case for EPIC-MOS.
     
     As customary for SAS tasks, each widget button or menu in the evselect window corresponds to a task parameter. The whole list of available evselect parameters, with their description, is available at the evselect task description.
     
     Once Run is clicked, dsplot is silently run on the created light curve, and the corresponding grace window appears (see Fig.4).
      
  3. Define a "low-background" interval on the light curve, using the following steps in the grace window:
      
     • go to the item Edit/Regions/Define
     • define region type (e.g.: in Horiz. Range)
     • click Define
     • click once with the left mouse on the grace window to define the starting time of the first interval, and once to define the end time.
     • repeat this as many times as needed. Just be aware that the counter Define region: needs to be updated if you want to define several regions (i.e., so as not to overwrite the old ones)
      
     
     Fig.4: A grace window, displaying a light curve, and one selection interval created as explained in the text
     Be aware that grace allows you to define several different types of intervals. Among the normally most useful: Above/Below line, in Vert. Range, Out of Horiz/Vert. Range, Left/Right of line
     
     When you are happy with your definition, click the button 1D region in the xmmselect window. The selection region will be automatically transferred into the data selection widget of the xmmselect window, and properly translated into a selectlib expression.
      
  4. Produce the filter event file by clicking Filtered Table in the main xmmselect window. This will bring up again the evselect window (see Fig.2). Here:
      
     • Mark the keepfilteredoutput option
     • Define the output file name in the filteredset widget (e.g.: EPICclean.fits)
     • Add to the  data selection widget of the xmmselect window:
       
           #XMMEA_EM &&(PI>150) for EPIC-MOS
           #XMMEA_EP &&(PI>150) for EPIC-pn
        
     • click Run

      

  5. The file EPICclean.fits contains the filtered EPIC event list and can now be used to produce scientific products. The procedure described using the xmmselect GUI, does not produce the EPICgti.fits. If one wants to produce this file for further use, simply run the command:
     
          tabgtigen table=rateEPIC.fits expression='Time_Expression' gtiset=EPICgti.fits
     
     where the Time_Expression is the time selection extracted from grace and used to create the file EPICclean.fits. This expression can be taken from the Filtering panel of the evselect window (see Fig.2).

An alternative way to produce an EPIC filtered event list and corresponding GTI file

The SAS task espfilt, one of the tools within the XMM-ESAS package (available as of SAS v9.0), is able to perform GTI filtering of EPIC event files. The task generates soft proton contamination-filtered products, including clean event lists and GTI files (refer to the espfilt documentation for a description of all the products generated and information on the task input parameters).

In its simplest form, the task can be ran by providing the name of the EPIC event list in the following way:

   espfilt eventfile=EPIC.fits

Relevant to this thread, three files are produced which contain the EPIC clean event list, GTI file and QDP text file of histograms and lightcurves. These files use the following naming convention:

   ****-allevc.fits, contains the EPIC clean event list for the field of view data
   ****-gti.fits, contains the definition of GTI
   ****-hist.qdp, contains QDP text file of histograms and lightcurves

where **** is a combination of the observation and exposure IDs. More files are produced by espfilt and the user is referred to the documentation of the SAS task. The QDP text file should be examined as it provides an indication of the quality of the data. It can be plotted using the command, for example, qdp ****-hist.qdp

Keep in mind that for the case of pn Small Window exposures, there are no corner events, hence, the products related to corner events will not be included as part of the output.