The Euclid Netherlands Science Data Centre

Nederland (met de RUG in het bijzonder) speelt een belangrijke rol in het beschikbaar maken van Euclid-data. Allereerst maakt Nederland, binnen het Euclid Netherlands Science Data Centre, net als acht andere lidstaten en ESA de data van de Euclid-satelliet beschikbaar voor onderzoekers. Daarbij heeft Nederland de archiefstructuur ontwikkeld waarmee alle lidstaten werken. Bovendien koppelt Nederland samen met Duitsland een enorme hoeveelheid gegevens van aardse telescopen aan het archief. Ook maakt Nederland met Italië de nabij-infrarood-afbeeldingen wetenschappelijk geschikt.

Euclid: the mission

Launched on 1 July 2023, ESA’s Euclid space telescope is dedicated to mapping the geometry of the Universe, shedding light on the enigmatic “Dark Universe”—comprising dark matter and dark energy, which constitute the majority of the cosmos' energy budget. Euclid specifically examines the distance-redshift relationship and the evolution of cosmic structures by measuring the shapes and redshifts of galaxies and galaxy clusters, reaching redshifts of ~2—equivalent to a look-back time of 10 billion years. Over its six-year mission, Euclid will cover the entire epoch during which dark energy played a crucial role in accelerating the Universe’s expansion.

Euclid Science Data Centres

The scientific processing of Euclid’s data—up to the production of final scientific measurements—is managed by the Science Ground Segment under the Euclid Consortium. This processing takes place within national Science Data Centers (SDCs, see figure below).

The Netherlands Science Data Centre (SDC-NL) plays a pivotal role in Euclid’s data processing and contributes in three other key areas:
• Development of the Euclid Archive System (EAS): A fundamental system supporting the processing, storage, and distribution of all Euclid data across the Science Ground Segment.
• Joint leadership of the Organizational Unit (OU-EXT): Responsible for integrating vast amounts of ground-based data with space-based observations to achieve Euclid’s scientific objectives.
• Deputy leadership of the Organizational Unit (OU-NIR): Overseeing the development of the Near Infrared Imaging Pipeline.

Euclid Netherlands Science Data Centre: Institutes and Facilities

The Netherlands Science Data Centre is hosted by the Kapteyn Astronomical Institute and the Centre for Information Technology (CIT) at the University of Groningen, with support from Leiden Observatory. By leveraging CIT’s existing computational infrastructure, SDC-NL achieves significant cost savings compared to maintaining dedicated systems. As of 2023, its resources include:

• 3 PB of storage in the CIT Data Handling System for bulk file storage.
• 80 TB of SSD storage and two ORACLE database servers in the CIT Database Infrastructure to store Euclid metadata.
• 8 servers in the Data Handling System to support Euclid Archive System services.
• 600 dedicated cores in the CIT Habrok High Performance Cluster for computational tasks.

Euclid Archive System

Developed collaboratively by SDC-NL and the European Space Astronomy Centre (ESAC), the Euclid Archive System (EAS) forms the backbone of the Euclid Science Ground Segment. This advanced, data-centric scientific information system manages up to 100 PB of mission data in a heterogeneous storage environment, enabling extensive access to both data and metadata throughout the mission. EAS comprises three key components:

• Euclid Archive Data Processing System (EAS-DPS): Developed by SDC-NL, this system stores all metadata defined by the Euclid Common Data Model, centralizing data processing tracking within the Science Ground Segment.
• Euclid Archive Distributed Data System (EAS-DSS): Also developed by SDC-NL, this system handles data file storage, supporting services such as cut-out and visualization tools. It is distributed across national Science Data Centers, with at least one storage node per center.
• Euclid Archive Science Archive System (EAS-SAS): Developed by ESAC, this system stores scientific metadata—a subset of the metadata in EAS-DPS—optimized for scientific use, providing access for consortium members and public users.

Euclid External Data

To accurately determine galaxy photometric redshifts and stellar Spectral Energy Distributions (SEDs), Euclid’s Near Infrared survey must be combined with optical ground-based surveys. Additionally, many of Euclid’s science cases rely on integrating Euclid imagery with ground-based optical data to construct full SEDs for large object samples. To ensure consistency, all external datasets must undergo a standardization process—referred to as Euclidization—which aligns them with Euclid’s data in terms of:

• Photometry: Ensuring consistent inferred SEDs and absolute scaling,

• Intrinsic light distribution: accounting for optical point spread function (PSF) effects and other image distortions,

• Astrometry: Establishing a uniform reference system and calibration approach,
• Data format and content (consistent characterization and correction of instrumental fingerprints, error propagation, absolute and relative unit systems, etc.).
  

Euclid Near Infrared Imaging Pipeline

The Near Infrared (NIR) reduction pipeline processes Euclid’s raw imaging data in multiple stages, refining it to the level required for scientific analysis. Given the unprecedented data volume, this process is fully automated and relies on rigorously tested algorithms. Precision and accuracy is paramount, as much of Euclid’s cosmological research depends on photometric redshifts, which demand exceptionally accurate and reliable photometry. The Netherlands serves as deputy lead of the NIR pipeline, responsible for defining, designing, and testing the pipeline, as well as spearheading the development of critical algorithms and modules for key processing stages.