NOBUGS 2024

Session

Data Reduction

Sep 25, 2024, 3:55 PM

Hybrid event (ESRF Auditorium)

Conveners

Data Reduction: Data Reduction

• Junrong Zhang (China Spallation Neutron Source)

51. Digital Twin Design and Implementation at the Institut Laue-Langevin

Shervin Nourbakhsh (Institut Laue-Langevin)

9/25/24, 3:55 PM

Data Reduction

Talk

Data from virtual experiments are becoming a valuable asset for research infrastructures: to develop and optimize current and future instruments; to train in the usage of the instrument control system; to study quantifying and reducing instrumental effects on acquired data. Furthermore large sets of simulated data are also a necessary ingredient for the development of surrogate models...

58. Dranspose: A constrained map-reduce live analysis pipeline for fast experimental feedback

Dr Felix Engelmann (Max IV, Lund University)

9/25/24, 4:10 PM

Data Reduction

Talk

During beamtimes, critical decisions on how to proceed with an experiment must be made constantly. As a result, it is important to provide feedback with the best possible data analysis, mostly in the form of visualizations, with the lowest possible latency. For low data rates, writing and monitoring a file works well. However, processing tens of gigabytes per second is difficult with a...

59. Integrated Sample Management, Data Acquisition and Processing for MX at the Canadian Light Source

Michel Fodje (Canadian Light Source, Inc)

9/25/24, 4:25 PM

Data Reduction

Talk

The Canadian Macromolecular Crystallography Facility (CMCF) consists of two beamlines (CMCF-ID and CMCF-BM). The beamlines are operated through a modern computer software system for on-site and remote collection. It consists of a user-friendly graphical user interface for experiment-focused data collection (MxDC), a laboratory information management system for remote planning, experiment...

140. ASAPO: A high-speed streaming framework to support an automated data-processing pipeline.

Mikhail Karnevskiy (DESY)

9/25/24, 4:40 PM

Data Reduction

Talk

Modern scientific experiments often generate large amounts of data, posing challenges for real-time
processing and analysis. ASAPO, a high-performance streaming framework developed at DESY,
addresses these challenges by providing a robust solution for online and offline data processing.
Leveraging TCP/IP and RDMA over Ethernet and Infiniband, ASAPO facilitates high-bandwidth
communication...

86. The High-throughput Data I/O framework for HEPS

Shiyuan Fu (IHEP)

9/25/24, 4:55 PM

Data Reduction

Talk

The High Energy Photon Source (HEPS) is estimated to produce a substantial volume of raw data, presenting significant computational challenges in scientific research. To address this problem, we have developed a high-throughput data I/O framework specifically tailored for HEPS, aimed at mitigating the I/O bottlenecks.Firstly, within this framework, we have devised a unified I/O interface for...

106. DAMNIT

Thomas Michelat (European XFEL)

9/25/24, 5:10 PM

Data Reduction

Talk

The Data And Metadata iNspection Interactive Thing (DAMNIT) is a tool developed by the Data Analysis group at the European XFEL (EuXFEL) to help scientists and users effortlessly create overviews of their experiments.

Traditionally, at EuXFEL, many user groups and beamline scientists use spreadsheets and electronic logbooks to track experimental settings, metadata, and analysis results....

98. Fast GPU-friendly indexing (cell reorientation) algorithm for X-ray crystallography.

Hans Christian Stadler Kleeb (Paul Scherrer Institute)

9/25/24, 5:10 PM

Data Reduction

Talk

Authors: Piero Gasparotto, Luis Barba, Hans-Christian Stadler,
Greta Assmann, Henrique Mendonça, Alun W. Ashton, Markus Janousch,
Filip Leonarski and Benjamı́n Béjar

TORO (TOrch-powered Robust Optimization) [1] is a new algorithm
for indexing diffraction patterns, applicable when the unit cell
geometry is known. Originally based on the PyTorch framework a
dedicated version in CUDA has...