contributing your data to ESASky

Contact us if you want to make your data available through the ESASky interface.

To prepare your data, you'll have to follow several steps:

1. For the imaging and spectra data:

• Generate the all-sky views (HiPS) for your data collection (optional).
• Create a metadata table with all the information on the observations, ideally following the ObsCore IVOA standard (for a very useful ressource, see the UCD Builder tool from CDS).  

2. For the catalogue data:

• Provide the full catalogue data and the list of columns to be displayed in ESASky.

adding your imaging AND/OR Spectral data to esasky

HIPS GENERATION

For the visualization of your data on ESASky, you can provide at least one Hierarchical Progressive Survey (HiPS) map displaying all your observations. These maps are generated using the Aladin/Hipsgen software as explained in this CDS tutorial. Depending on the nature and heterogeneity of your data, it may be necessary to perform some sort of pre-processing on your data (e.g. to homogeneize image backgrounds) prior to the HiPS generation.

You can provide a map for every instrument or band in your survey, or just a color RGB composition. Each HiPS must be available at least in JPEG or PNG format; the FITS format is not strictly required by the interface, but it is a necessary step to generate a HiPS from FITS images, and it may be interesting for some users to have access to it in order to inspect the data more closely.

The HiPS generation code also produces a Multi-Order Coverage map (MOC) for each created HiPS. We only require one MOC for the whole survey, which can be generated from the complete observation collection or by joining the MOCs from individual instruments/bands.  

Each HiPS is accompanied by a 'properties' text file including basic information on the map such as: observations used, observations source and copyright, version of the HiPS generation code, parameter values used in the HiPS generation. This file is created automatically by the HiPS generation code and can be edited to add the necessary information. In particular, an IVORN identificator and a URL for the HiPS retrieval must be indicated, and the HiPS status must be 'public master clonable', so that we can mirror it in our servers (in order to mitigate the risk of having too many queries sent to your server by ESASky users). 

In addition to the properties file for every individual HiPS, we require a plain text file summarizing the basic information on the HiPS collection, so that we can include it in the HiPS description section of these help pages. Concretely, we require the following:

Mission/Survey name: 

Click here to download a template for this file.

OBSERVATIONS METADATA

Allowing the users to inspect and download high-level products from a specific mission requires providing all the needed metadata associated to the  observations. This information will be stored as a table in the ESASky postgres database. CSV is the most suitable table format to be used for ingestion in the system.

The compulsory columns the corresponding mission observation table should provide are described below:

• Observation_id: Unique observation identifier, given by the mission archive.
• RA (degrees)
• Dec (degrees)
• Instrument: Name of the instrument related to the observation
• Filter: Name of the filter / energy band related to the observation
• Start_UTC: Start time of observation in UTC ISO8601 format (yyyy-mm-dd hh:mm:ss)
• End_UTC: End time of observation in UTC ISO8601 format (yyyy-mm-dd hh:mm:ss)
• Duration: Exposure time (seconds)
• Product_url: URL/path to download the observation high-level product FITS image.
• STC-s: Footprint of the observation in STC string format (STC-s). This is the standard format used by ESASky to display polygons and calculate individual Field Of Views parsing the STC-s string line into a pgsphere function. We use the STC-s footprint as input for this field, but in case more than one polygon is provided per observation, it should be notified in order to generate accurate FOV.

Optional information:

• Postcard_url: URL to the imaging data preview, when the mission archive provides preview images (jpeg, png) of the high-level products accesible publicly.
• Archive_url: URL/path to open the archive with results for the specific observations.

Any additional metadata that you may consider relevant to be included in ESASky apart from the mentioned fields above.

FOOTPRINT GENERATION

For the generation of the observation footprints from the high-level imaging products, we recommend the use of the python script footprintfinder, developed by the Space Telescope European Coordinating Facility (ST-ECF). Please visit the official site for download and further information.

The footprintfinder script allows the following input command line options:

 -e, --extension      FITS file extension to use. Default=first
                        extension with data.

   Processing:
   -t, --tolerance      Maximum tolerated distance of a pixel from
                        a polygon line. Default=5.
   -z, --zerovalue      Value of the pixels that should be
                        identified as 'border'. If a pixel value
                        'nan', it is replaced with the zerovalue,
                        too.
   -m, --minlength      Minimal length of a continous footprint
                        border to be taken into account as a
                        fraction of the total number of border
                        pixels. Default=0.05.
   Output:
   -p, --plotting       Plots the footprint. Default=off.
   -d, --ds9            Writes output file(s) in ds9 region format.
                        One file in image coordinates and if possible
                        (i.e. projection is RA--TAN, DEC-TAN) a
                        second one in WCS coordinates are produced.
                        Default=off.
   -w, --writepoints    Writes the polygon coordinates on the screen.
                        Default=off.
   -v, --verbose        Increase verbosity.

   Help:
   -h,  --help          Print this help.

In case you have any problems with the footprint generation please contact us. Even though the footprint finder python script is a very good solution, there may be cases (restricting number of polygons, low vs high number of points per polygons, computing time ...) where a different approach should be implemented, as a geometrical reconstruction of the instrumental FoV.

adding your catalogue data to esasky

In order to include a catalogue in the application, we require the full published catalogue file or the location to retrieve it. As in the case of observation metadata, the full catalogue is ingested and stored locally in the ESASky database.

The catalogue category is quite inhomogeneous across diferent energy bands and the type of catalogue itself, and therefore we customize the metadata displayed in the application individually (for more information, visit ESASky Catalogues help page). However, there are three common fields that we require in order to display it on the sky:

• Source name
• Coordinates (RA, Dec) in degrees.

Additional information, depending on the catalogue source may be:

• ra_err
• dec_err
• proper_motion_ra
• proper_motion_dec
• magnitude
• flux (specifying wavelength/energy regime)
• ...

Please send us the fields/columns from your particular catalogue that are relevant for inclusion in ESASky result panel. The input should also describe the fields and units that should be displayed in the application with the associated column names from the catalogue (i.e. "flux_1" original catalogue column name => Flux__(0.2-2keV) [ergs*s^_-1cm^-2] displayed in the application).