IoW_20160610 - Gaia

Image of the Week
Run 1000 of the Initial Data Treatment system

	Figure 1:IDT web-based front-end tool showing the 1000th IDT run. Click here to see how the monitoring system behaves during a few hours of the mission.
Yesterday, almost two years after the start of Gaia's nominal operations, the Initial Data Treatment system (IDT) finished its 1000th run - a remarkable milestone for this critical system in charge of the daily data processing in the Gaia Data Processing and Analysis Consortium (DPAC). The IDT software is executed on some 28 computing nodes at the Gaia Science Operations Centre (SOC, at ESAC, near Madrid), receiving raw data from the spacecraft through the Mission Operations Centre (MOC, in Darmstadt), the MIT (MOC-SOC Interface Task), and the DCS (Decompression and Calibration Service). It processes spacecraft attitude data, astrometric, photometric, and spectroscopic instrument data, Basic Angle Monitor data, Wave Front Sensor data, and a handful of ancillary data packet types. Additionally, it runs complex algorithms to determine "intermediate data": refined attitude, basic angle variations, sky background, image parameters (fluxes, positions, quality, etc.) and a first cross-matching (identification of measurements in the reference on-ground catalogue). These algorithms require several instrumental calibrations determined either by IDT itself or by other DPAC systems. All this must be done in real-time, on a daily basis throughout the whole mission, to feed the so-called First Look diagnostics and calibration system, as well as all the downstream Data Processing Centres (DPCs). Thus, IDT feeds the whole Gaia consortium. The Initial Data Treatment works in "runs", which is a way to handle and identify a bunch of data measured and processed during a given time period. Typically, IDT does one run about every 23 hours of observations (there is no need to have exactly one IDT run per day). This allows enough data to be accumulated to get self-consistent outputs and perform representative diagnostics. During the first months of the mission, the IDT runs covered shorter time ranges to speed up some of these diagnostics and assess the health of the satellite and on-ground data processing systems. IDT has its own diagnostics, generating over 9000 plots and statistics every day to determine the data quality and trends, allowing to detect potential inconsistencies or issues before they may become a problem (either on-board or on-ground). All this is accessible through a web-based front-end, but also PDF reports are automatically generated at the end of every run. Figure 1 shows the main page of the tool, which shows the moment when Run 1000 was closed. One way to identify Gaia measurements is their acquisition time in 'mission revolutions' (spins of Gaia around its rotation axis). The four main plots of Figure 1 illustrate the mission time (vertical axes) of the several data types being received or processed at a given wallclock time (horizontal axes). Here IDT was recovering from a downtime to perform some maintenance, quickly reaching again the 'real-time line' (the dashed diagonal line on each plot). Vertical dashed lines indicate the Runs done. The progress indicators at the top show the numbers of 'Elementaries' (intermediate data outputs, one per measurement), the system status, the fraction of Runs done, the pending processing tasks, or the number of computing threads being used.

		Figure 2: Sky-coverage of a typical IDT run. Click on the image for a high-res version.
To illustrate the daily workload and accuracy of the IDT system and Gaia, Figure 2 draws the equatorial sky coordinates (right ascension and declination), as well as the onboard-estimated brightnesses, of all objects detected, measured, downlinked and processed during a typical day, having "only" 43 million objects in this case (some days over a hundred million objects can be reached). The sky coordinates are those determined by IDT in real-time, which are already very accurate (around 100 milliarcseconds, or 1/36000th of a degree). The original, artificial image has 603 megapixels which not all image viewers support, which is why a downsized version is provided here. However, even the original image size would not be enough to illustrate the resolution in high sky densities, that is, when observing the Galactic Plane. When zooming in, some features can be seen, including the "strips" of several scans of Gaia on the sky (one scanning revolution every 6 hours).

		Figure 3: Zoom of Figure 2. Click on the image for a high-res version.
Figure 3 shows a zoom into the southern region, where some sky features and structures can be seen, as well as some further "strips" corresponding, in this case, to the seven CCD rows (or "cameras") of the Gaia instruments.

		Figure 4: Zoom of Figure 3. Click on the image for a high-res version.
Finally, Figure 4 shows another zoom (with increased dot sizes for better visibility) on a couple of globular clusters, NGC 1831 and NGC 1866, the individual stars of which are detected and measured by Gaia without any problem. Only at this zoom level, individual objects as detected by Gaia can be seen. Full-resolution images of Figures 2 and 3 can be downloaded here.
Credits: ESA/Gaia/DPAC/UB/IEEC [Published: 10/06/2016]