17-18 September 2019
ESRF (Grenoble)
Europe/Paris timezone
Home > Contribution List

Contribution List

Displaying 42 contributions out of 42
Presented by Arthur PETITPIERRE
Type: 30' Talk Session: HDF5 - Industrial use cases
C-Blosc2 is the new major version of C-Blosc, with a revamped API and support for new compressors and new filters (data transformations), including filter pipelining, that is, the capability to apply different filters during the same compression pipeline, allowing for more adaptability to the data to be compressed. Dictionaries are also introduced, allowing better handling of redundancies among in ... More
Presented by Alted FRANCESC on 18 Sep 2019 at 09:30
Type: 20' Talk Session: HDF5 - Science use cases
The Coherent Imaging Division of the Center for Free Electon Laser Science develops innovative methods for imaging using X-ray Free Electron Laser (XFEL) and synchrotron sources, with an emphasis on bioparticles and macromolecules. The HDF5 file format has in recent years become the de-facto standard data storage format in this field. Modern x-ray detectors like the AGIPD, EIGER and JUNGFRAU, ... More
Presented by Dr. Valerio MARIANI on 17 Sep 2019 at 14:00
The HDF Group has been developing and supporting HDF technologies since 1988. Our mission has always been to support the HDF user community and to insure that the HDF ecosystem can evolve to meet the ever-changing world of data. In this talk we will give a brief history of HDF and the HDF Group; explain our mission and commitment to the data community; describe our culture and business model and c ... More
Presented by Dr. Michael FOLK on 17 Sep 2019 at 09:00
HSDS (Highly Scalable Data Server) is a web service for HDF data. By enabling HDF functionality over a REST API, HSDS can provide a solution for many applications which would be difficult to implement with just the HDF library. In this session we will cover: 1. Motivation for a service-based implementation of HDF 2. HSDS Features 3. Architectural Design 4. Storage Schema (and comparison w ... More
Presented by Mr. John READEY on 17 Sep 2019 at 11:00
silx is a Python library initiated by ESRF to support the development of data assessment, reduction and analysis applications. It provides support for different file formats, data reduction routines including GPU-based ones, [Py]Qt widgets to browse and visualize data. One of its purpose it to smooth the transition from the previous data acquisition file formats used at ESRF (SPEC + EDF) to HD ... More
Presented by Dr. Thomas VINCENT on 18 Sep 2019 at 15:10
Presented by Mr. Jan KOTANSKI
Presented by Vicente Armando SOLE
Presented by Mr. sylvain FERRIOL, Dr. Clément BUTON
Presented by Stéphane POIRIER, Dr. Majid OUNSY
Presented by Simon EBNER
Presented by Dr. Loïc HUDER
For more than two decades The HDF Group developers have been working with researchers all over the globe helping to capture, store and analyze experimental data in HDF5. In the past several years, the amount of experimental and observational data stored in HDF5 and the rate at which this data is collected have created new challenges for the scientists and triggered requests for new features in HDF ... More
Presented by Ms. Elena Pourmal POURMAL on 17 Sep 2019 at 09:30
The HDF Group members will hold a round table session to discuss the issues brought by the HDF5 Workshop participants. The HDF5 Workshop is a great venue to share any items on your HDF5 wish list, to ask technical questions, and to have a discussion with fellow participants. Please submit your discussion item(s) using this [Web form][1] before you come to Grenoble. Knowing the issues in advanc ... More
Presented by Ms. Elena Pourmal POURMAL, Dr. Michael FOLK, Mr. John READEY on 18 Sep 2019 at 11:00
At the MAX IV Laboratory we have developed a web based viewer for HDF5 data files. We designed it to be simple and fast, with the aim of allowing users to quickly and conveniently browse their HDF5 data files and visualize their data sets. I will discuss the structure and features of the application and give a quick demo.
Presented by Dr. Jason BRUDVIK on 18 Sep 2019 at 14:50
[NeXus][1] is a data model that promotes standardisation of the organisation of scientific information. It is now commonly used in the X-ray, neutron and muon communitities, and is readily extendable to more diverse applications after two decades of maturating. NeXus currently recommends HDF5 as the container format for NeXus data and as such, the capabilities and limitations of HDF5 have influenc ... More
Presented by Dr. Benjamin WATTS on 17 Sep 2019 at 14:20
Presented by Dr. Benjamin WATTS
While support of the HDF5 format in ESRF applications dates back to late 2009, ESRF is basically the last synchrotron radiation facility adopting HDF5 as main data format for acquisition. This talk presents the ideas behind the implementation of HDF5 and NeXus at the ESRF in the context of data acquisition, data analysis and data policy. The status of the implementation will also be presented.
Presented by Dr. Vicente Armando SOLE JOVER on 17 Sep 2019 at 14:40
Type: 20' Talk Session: HDF5 - Science use cases
Present the current usage of HDF5 at ITER for data archiving
Presented by Dr. lana ABADIE on 17 Sep 2019 at 16:50
Type: 20' Talk Session: HDF5 - Applications and Tools
Rust is a modern system programming language focused both on memory safety and high performance. [hdf5-rust](https://github.com/aldanor/hdf5-rust) is an open source project aiming at providing threadsafe and memory-safe high-level Rust API for most of the HDF5 functionality alongside the low-level bindings. In this talk we will discuss, among other things, how particular HDF5 concepts can be ... More
Presented by Dr. Ivan SMIRNOV on 18 Sep 2019 at 16:20
The Earth's magnetic field is generated by motions of the Earth's liquid core, a process called the geodynamo. The geodynamo problem is so challenging that even recent numerical simulations fail to capture the features observed in the geomagnetic field. Data assimilation algorithms appear to be a viable way to tackle the geodynamo problem. Such algorithms incorporate information from both numerica ... More
Presented by Dr. Loïc HUDER on 17 Sep 2019 at 16:10
Presented by Ms. Elena Pourmal POURMAL
Presented by Mike FOLK
Type: 20' Talk Session: HDF5 - Science use cases
Diamond Light Source uses HDF5 in its data acquisition to write NeXus files and it is an essential component for our data analysis. We have (co-)sponsored some advanced features that debuted in version 1.10 including SWMR and VDS. These features are used across a number of experimental stations in data acquisition systems including GDA, EPICS (Area Detector) and Odin (OdinData). Both online a ... More
Presented by Dr. Peter CHANG on 17 Sep 2019 at 15:00
Presented by Dr. Jason BRUDVIK
Type: 30' Talk Session: HDF5 - Science use cases
* Presentation of past and future collaborations between HDF Group and AWS * Showing the advantages of infrastructure as code applied to HPC workloads * Presentation of EFA (Elastic Fabric Adapter), AWS low-latency interconnect and how it can benefit to HPC workloads in general and HDF5 in particular * How to experiment with HDF5 on AWS
Presented by Mr. Arthur PETITPIERRE on 17 Sep 2019 at 17:10
Type: 20' Talk Session: HDF5 - Applications and Tools
This contribution presents how the gzip compression engine integrated into the IBM Power9 processor can be used with the HDF5 storage to provide faster saving of actual diffraction data compared to the software compression. The default compression provided as part of HDF5 will be compared with meta-compressors like Blosc. A few pre-compression filters will be presented and their performances co ... More
Presented by Dr. Jerome KIEFFER on 18 Sep 2019 at 14:00
Type: 20' Talk Session: HDF5 - Science use cases
At DESY metadata and data are stored in HDF5 files using the NeXus format. Our NeXus framework is based on c++ libraries (h5cpp and PNI) as well as python bindings (h5py and h5cpp). It gains a modular structure with a help of Tango Controls software. The NeXus configuration is stored in MySQL DB in parts called components, i.e. XML NXDL-like-strings. They are created by nxstools scripts from ... More
Presented by Dr. Jan KOTANSKI on 17 Sep 2019 at 16:30
Nexus/HDF5 data format is the “standard” on all our beamlines since the beginning of operations at SOLEIL. Our scientists were reluctant to adopt at first, but are now recognizing that it allows them to retrieve old data with their context. This was recognized at a workshop given during a SOLEIL users’ meeting in 2015. This presentation will describe the different steps that have been worke ... More
Presented by Stéphane POIRIER on 17 Sep 2019 at 15:20
Type: 30' Talk Session: HDF5 - latest developments
OpenIO SDS (Software Designed Storage) is an open source object storage platform that can provide a performance and cost-effective means of storing large data collections. As an alternative to parallel file systems or NAS storage solutions, OpenIO SDS provides features such as: • Built in redundancy • Scalability to PB sized collections • Ease of expanding storage • Use of any heter ... More
Presented by Mr. Jean-Francois SMIGIELSKI, Mr. John READEY on 17 Sep 2019 at 12:00
This talk is a brief outline of real-world daily use of HDF5 in finance, with discussion of both the strong and weak points of the library and its Python/C++ wrappers, along with a few suggestions on what could be enhanced to improve general usability.
Presented by Dr. Ivan SMIRNOV on 18 Sep 2019 at 09:00
Presented by Benjamin WOOLFORD-LIM
This talk describes the Allotrope Framework, a new data standard for data in life sciences. Built on HDF5 technology, the Allotrope Framework is made to store experimental data and metadata in a high-performance format suitable for a variety of scientific techniques. The Framework leverages Semantic Web approaches, audit trailing, and checksums to enable coherent, consistent, and interoperable dat ... More
Presented by Mr. Benjamin WOOLFORD-LIM on 18 Sep 2019 at 10:00
Adobe’s [Extensible Metadata Platform][1] (XMP) is an open standard (ISO 16684-1) for attaching metadata to files. It is used by such ubiquitous formats as JPEG, MP3 and PDF, and most computer operating systems include components for reading and displaying this metadata (including thumbnail images) in their file browsers. This functionality is pretty much taken for granted nowadays for image fil ... More
Presented by Dr. Benjamin WATTS on 18 Sep 2019 at 16:00
Presented by Ms. Elena Pourmal POURMAL
Presented by Dr. Thomas CASWELL on 18 Sep 2019 at 14:20