Speaker
Description
The information technology (IT) requirements of complex data analysis have been growing steadily in the last decades. Among the techniques, readily performed at synchrotrons, computational tomography (CT) is one of the most IT resource demanding. This holds both for the computing (CPU and GPU) and storage (I/O) requirements. Taking into account the faster and larger detectors (exceeding 5 megapixels in resolution) and detailed scans (several thousands of projections), the individual datasets readily take several tens of gigabytes of space and thus result in total reconstruction times of tens of minutes in certain cases. Traditionally, tomography workflows have focused most of their attention on speeding up the said reconstruction process. These efforts have resulted in capable and portable open source libraries and packages such as ASTRA and TomoPy. At SYRMEP beamline of Elettra Sincrotrone Trieste, we have recently started to rethink the user interaction with the beamline (specifically the beamline’s IT systems and its reconstruction software STP [1, 2]), with a view towards the new Life science beamline, which will be constructed during the ongoing accelerator upgrade (Elettra 2.0 project). Firstly, we have taken into account the actual user’s full workflow. We are now considering the full data pipeline, including the interaction of the user with the data itself, i.e. access, loading and saving of the images or when the user is refining the phase retrieval or reconstruction parameters. This not only calls for efficient algorithms, but also for a modern interactive user interface to the remote computing resources, allowing for multiple different CT data processing workflows to be used.
We present here for the first time our work on a novel framework which allows for simultaneous interactive work and working with HPC-level resources. In brief, the total time of a standard procedure from acquisition to the final full reconstructed volume is reduced from several hours down to tens of minutes. Users are able to interactively refine the parameters using a modern web-based interface which allows for seamless remote work and is able to access the storage in a fast and efficient manner. The actual workflow is not imposed on the user, but a collection of ways to interact with the system exist. For example, a user can choose to refine the reconstruction parameters in-program or obtain them from another source, saving them into the framework’s database and then running the jobs in batch mode. Another option is to do each individual full reconstruction directly after obtaining the parameters, bypassing the batch mode and the database entirely. Additional workflows are supported, as the framework’s components (backend and frontend) are interoperable and independent of each other.
All this has benefits both on the user, as well as the beamline side. For the user, the whole data analysis process is significantly faster (working on our HPC-grade resources without prior reservation or queue system) and more efficient, where the reconstructed images are obtained already during the beamtime. For the beamline, this opens up opportunities for performing near real-time reconstruction, seamlessly linking the data analysis, acquisition and storage systems.
[1] F. Brun et al., (2015) Fundamenta Informaticae, 141 (2-3), pp. 233-243, DOI: 10.3233/FI-2015-1273
[2] F. Brun et al., (2017) Advanced Structural and Chemical Imaging 3:4, DOI: 10.1186/s40679-016-0036-8
| Abstract publication | I agree that the abstract will be published on the web site |
|---|
