Sep 23 – 27, 2024
ESRF Auditorium
Europe/Paris timezone

The Design of HDF5 Data Formats for HEPS

Sep 24, 2024, 6:00 PM
2h
ESRF Entrance Hall

ESRF Entrance Hall

Poster Metadata Posters

Speaker

Haofan Wang (Institute of High Energy Physics, CAS)

Description

The High Energy Photon Source (HEPS) encompasses a variety of experimental types, including diffraction, scattering, imaging, and spectroscopy. The data generated from these experiments are highly dimensional, uncertain, and computationally complex. Considering the users' needs for interoperable data analysis and high-performance I/O processing, it is necessary to organize and manage the data and metadata in a rational manner. We have adopted HDF5 and NeXus as the unified data format standards for HEPS. Sequentially interfacing with the first phase of 14 beamlines, we have organized the data formats and summarized common requirements, eventually developing three generic data formats. These serve as foundational templates for the subsequent phases of 90 beamline stations.

Initially, the report will introduce the characteristics of experimental data types at HEPS and the current status of experimental data formats. From the perspectives of experimental users and beamline station managers, we will detail the organization and specification requirements of various experimental data HDF5 formats designed according to NeXus. Furthermore, we will demonstrate the data format organization schemes with examples from the B2 Hard X-Ray Nanoprobe Multimodal Imaging Beamline and the B7 Hard X-Ray Imaging Beamline, both of which are high-throughput stations. Finally, we will summarize some challenges and future plans concerning HEPS data formats.

Abstract publication I agree that the abstract will be published on the web site

Primary authors

Fazhi Qi (Institute of High Energy Physics, CAS) Hao Hu (Institute of High Energy Physics, Chinese Academy of Sciences) Haofan Wang (Institute of High Energy Physics, CAS) Qi Luo (Institute of High Energy Physics, CAS)

Presentation materials