Figure 1: A screenshot taken from the Gaia Archive showing the new feature "external TAP search". Image credit: ESA/ESDC

The release of the second batch of Gaia data on 25 April 2018 came along with substantial improvements to the Gaia Archive, aimed at facing the great challenge of handling the nearly 20,000 GB of data hosted in it (including not only Gaia DR2 data, but also the previous data releases and ancillary data as well as the data stored in user spaces). These improvements also prepared for Gaia's future and even more challenging data releases.

Over one year of Gaia DR2 data, archive users have performed around 62 million queries and produced more than 1000 refereed papers. In the image below you can see an overview of the amount of different IP addresses downloading Gaia data per month with a clear jump near the release date of Gaia DR2 on 25 April. Data downloaded before this date was data from the first Gaia Data Release.

Figure 2: Number of active IP addresses per month since September 2016 (when Gaia DR1 was released) through April 2018 (when Gaia DR2 was released) up to April 2019 (one year after Gaia DR2). Image credit: ESA/ESDC

Figure 3: Amount of Gigabytes downloaded per month since September 2016 (when Gaia DR1 was released) up to April 2019, which is one year after the release of Gaia DR2. Gaia DR2 was released in April 2018, which is clearly visible in the graph showing a high peak in downloaded data in the period right after the release of Gaia DR2. Image credit: ESA/ESDC

The Gaia Archive team is already working to get everything ready for the next data releases, Gaia EDR3 (third quarter of 2020) and Gaia DR3 (second half of 2021). The processing of the data that will lead to Gaia EDR3 and Gaia DR3 is currently ongoing in the Gaia Data Processing and Analysis Consortium. In the meantime, users of the archive continue exploring Gaia DR2 data and can get support via the Gaia Helpdesk.

However, even in between Gaia data releases, archive developments are ongoing. New versions of the Gaia Archive are put out regularly and a new version was released in May 2019. This new version provides you with an upgraded performance and new features.

One of these new features is interesting for scientists wishing to combine data from the literature or from other data providers with the catalogues hosted in the Gaia Archive. They now have a faster and more efficient way to do so: it is now possible to connect to an external data server from inside the Gaia archive, query the tables hosted in it and work with them within the user’s personal space. The only condition is that the server is compliant with the Table Access Protocol (TAP) standard, a Virtual Observatory (VO) protocol for accessing general table data, including astronomical catalogs as well as general database tables. Access is provided for both database and table metadata as well as for actual table data.
Tables from external TAP queries are stored in a different workspace in the archive, where the user can manipulate them with ADQL queries as they would do with any table directly hosted in the archive. The tables resulting from these jobs can also be uploaded, if wanted, as new user tables in the user’s Gaia workspace. To learn more, watch the below video tutorial or refer to the Gaia Archive help. Users needing support to get started with this functionality can contact the Gaia Archive team via the Gaia Helpdesk.

This video shows how to connect to an external TAP server from inside the user's space in the Gaia Archive. Credits: ESA/ESDC

Another new feature of the Gaia Archive is the inclusion of the RUWE data. A recipe was already provided through the Gaia DR2 known issues page, and some of the Gaia partner data centres already served the RUWE data along with the Gaia data. A decision was made in the Gaia Data Processing and Analysis Consortium to make an official release of the RUWE data along with the Gaia DR2 data, to ensure consistency of the RUWE data provided through ESA/Gaia/DPAC. The difference between the RUWE data as provided now through the Gaia Archive (and soon through the Gaia Partner Data Centres) and the RUWE data as was provided initially through for example the ARI partner data centre is reasonably small.

The above story is based on some of the news items that appeared in the ESDC newsletter on the Gaia Archive. If you are interested in following up on Gaia Archive developments and on all archive developments in general, this newsletter might be of interest to you. Of course there is also the Gaia Newsletter, dedicated to keep you updated on all Gaia related topics. More info on the Gaia newsletter can be found here.