Image of the Week

Matching observations to sources for Gaia DR4

Figure 1: Sky map, in galactic coordinates, illustrating the density of measurements processed by the recent cross-match run of IDU. The plot contains the 142 billion detections processed at the Data Processing Centre of Barcelona using a total of 750,000 CPU hours of the MareNostrum supercomputer. Here, the Galactic plane and the two Magellanic Clouds can clearly be seen, as well as other galaxies and globular clusters. Also easily recognizable are the typical patterns from the Gaia spacecraft scanning law.

While the scientific community is analysing the first batch of the Gaia DR3 data, part of the Data Processing and Analysis Consortium (DPAC) is already engaged in the production of Data Release 4. The Intermediate Data Update (IDU) pipeline and the Data Processing Centre of Barcelona delivered the results of the first cross-match run in preparation for the fourth Gaia data release. This is the first DPAC system that has processed, as a preparation for a catalogue release, the complete nominal mission – covering 66 months (about 5.5 years) of data and 142 billion detections.

The IDU system processes all on-board transit detections and assigns a unique source identifier for each cluster of detections that belong to the same astrophysical source on the sky. Therefore, this process builds the list of sources that will be treated in all subsequent, down-stream pipelines. The first stage of the cross-match run is the detection classifier to discriminate real from spurious detections, mainly caused by optical diffraction artefacts around bright objects.

Then the sky is partitioned into groups of detections and sources. These groups are then processed, without using any source information, to determine clusters of detections that belong to the same source. Finally, each cluster is assigned to one of the input catalogue sources, or a new source is created when needed.

The most remarkable non-astrophysical artefacts of the image are caused by the Gaia scanning law. These scanning law features are due to the fact that the number of matched detections of each source is not uniform. It is also noted that many of the compact bright points (away from scanning law features) are globular clusters or galaxies. Specifically, Figure 2 shows a zoom-in of the main sky map visualisation (Figure 1) and highlights the galaxy M31. The visualisation in Figure 1 excludes about 11.2% of the input detections identified as spurious detections and discarded from subsequent processing.

Figure 2: Zoom-in of the data visualisation in Figure 1 highlighting a region around the galaxy M31

In this first run for Gaia DR4, at least 50% of the sources were observed at least 44 times. It should be noted that this image has been constructed from intermediate data, as obtained after the run of the first pipelines (so before running the AGIS pipeline which determines the astrometric properties of the sources). It is remarkable to see that the sky map is already of high quality.

IDU uses the MareNostrum 4 supercomputer of the Barcelona Supercomputing Centre (BSC). For this cross-match run, a total of 750,000 CPU hours of the MareNostrum supercomputer were required, using up to 80 nodes. A second cross-match will run later this year using improved calibrations and will provide the final source list for the fourth Gaia data release.

Credits: ESA/Gaia/DPAC, F. Torra, J. Portell, J. Castañeda, M. Bernet, S. Bartolomé and CU3-IDU/DPCB team

[Published: 26/02/2021, added figure 2: 03/03/2021]