Speaker
Description
Google team showed a new Adam-tuned optimization solver for deep neural network training called LION (EvoLved Sign Momentum) by thousands of hours of training at TPU cluster, 2023. It is more memory-efficient than Adam as it only keeps track of the momentum and cuts the epsilon and a group of momentum parameters off. We applied “LION” solver to the physics-informed neural network-PtychoPINN which advanced one more step than PtychoNN from APS with physics constrained (probe matrix computed by ePIE first) in the loss function. The PtychoPINN algorithm is now used in High Energy Photon Source(HEPS), China, the fourth generation source with less than 20% GPU-memory occupied and 3X less time consumed.
Apart from the new LION solver, we moved this training process from Nvidia A100 to 8*Huawei Ascend 910A GPU which is called NPU(Neural Processing Unit). The CANN Mindspore toolkit were used and it could achieve 70% performance of A100. The training process would be revealed in the meeting.
| Abstract publication | I agree that the abstract will be published on the web site |
|---|
