Publications
Speeding up Madgraph5 aMC@NLO through CPU vectorization and GPU offloading: towards a first alpha release
The matrix element (ME) calculation in any Monte Carlo physics event generator is an ideal fit for implementing data parallelism with lockstep processing on GPUs and vector CPUs. For complex physics processes where the ME calculation is the computational bottleneck of event generation workflows, this can lead to large overall speedups by efficiently exploiting these hardware architectures, which are now largely underutilized in HEP. In this paper, we present the status of our work on the reengineering of the Madgraph5_aMC@NLO event generator at the time of the ACAT2022 conference. The progress achieved since our previous publication in the ICHEP2022 proceedings is discussed, for our implementations of the ME calculations in vectorized C++, in CUDA and in the SYCL framework, as well as in their integration into the existing MadEvent framework. The outlook towards a first alpha release of the software supporting QCD LO processes usable by the LHC experiments is also discussed.
The matrix element (ME) calculation in any Monte Carlo physics event generator is an ideal fit for implementing data parallelism with lockstep processing on GPUs and vector CPUs. For complex physics processes where the ME calculation is the computational bottleneck of event generation workflows, this can lead to large overall speedups by efficiently exploiting these hardware architectures, which are now largely underutilized in HEP. In this paper, we present the status of our work on the reengineering of the Madgraph5_aMC@NLO event generator at the time of the ACAT2022 conference. The progress achieved since our previous publication in the ICHEP2022 proceedings is discussed, for our implementations of the ME calculations in vectorized C++, in CUDA and in the SYCL framework, as well as in their integration into the existing MadEvent framework. The outlook towards a first alpha release of the software supporting QCD LO processes usable by the LHC experiments is also discussed.
|
Efficient phase-space generation for hadron collider event simulation
We present a simple yet efficient algorithm for phase-space integration at hadron colliders. Individual mappings consist of a single t-channel combined with any number of s-channel decays, and are constructed using diagrammatic information. The factorial growth in the number of channels is tamed by providing an option to limit the number of s-channel topologies. We provide a publicly available, parallelized code in C++ and test its performance in typical LHC scenarios. SciPost Phys. 15, 169 (2023) October 2023 Link to Paper |
|
Developments in Performance and Portability for MadGraph5_aMC@NLO
Event generators simulate particle interactions using Monte Carlo techniques, providing the primary connection between experiment and theory in experimental high energy physics. These software packages, which are the first step in the simulation worflow of collider experiments, represent approximately 5 to 20% of the annual WLCG usage for the ATLAS and CMS experiments. With computing architectures becoming more heterogeneous, it is important to ensure that these key software frameworks can be run on future systems, large and small. In this contribution, recent progress on porting and speeding up the Madgraph5_aMC@NLO event generator on hybrid architectures, i.e. CPU with GPU accelerators, is discussed. The main focus of this work has been in the calculation of scattering amplitudes and "matrix elements", which is the computational bottleneck of an event generation application. For physics processes limited to QCD leading order, the code generation toolkit has been expanded to produce matrix element calculations using C++ vector instructions on CPUs and using CUDA for NVidia GPUs, as well as using Alpaka, Kokkos and SYCL for multiple CPU and GPU architectures. Performance is reported in terms of matrix element calculations per time on NVidia, Intel, and AMD devices. The status and outlook for the integration of this work into a production release usable by the LHC experiments, with the same functionalities and very similar user interfaces as the current Fortran version, is also described. ICHEP2022 October 2022 Link to Paper |
|
|
|
Portability: A Necessary Approach for Future Scientific Software
Today's world of scientific software for High Energy Physics (HEP) is powered by x86 code, while the future will be much more reliant on accelerators like GPUs and FPGAs. The portable parallelization strategies (PPS) project of the High Energy Physics Center for Computational Excellence (HEP/CCE) is investigating solutions for portability techniques that will allow the coding of an algorithm once, and the ability to execute it on a variety of hardware products from many vendors, especially including accelerators. We think without these solutions, the scientific success of our experiments and endeavors is in danger, as software development could be expert driven and costly to be able to run on available hardware infrastructure. We think the best solution for the community would be an extension to the C++ standard with a very low entry bar for users, supporting all hardware forms and vendors. We are very far from that ideal though. We argue that in the future, as a community, we need to request and work on portability solutions and strive to reach this ideal. March 2022 Snowmass Contribution Link to Paper |
|
Porting CMS Heterogeneous Pixel Reconstruction to Kokkos
In this paper we use heterogeneous pixel reconstruction code from the CMS experiment at the CERNL LHC as a realistic use case of a GPU-targeting HEP reconstruction software, and report experience from prototyping a portable version of it using Kokkos. The development was done in a standalone program that attempts to model many of the complexities of a HEP data processing framework such as CMSSW. We also compare the achieved event processing throughput to the original CUDA code and a CPU version of it.
August 2021
Link to Paper
In this paper we use heterogeneous pixel reconstruction code from the CMS experiment at the CERNL LHC as a realistic use case of a GPU-targeting HEP reconstruction software, and report experience from prototyping a portable version of it using Kokkos. The development was done in a standalone program that attempts to model many of the complexities of a HEP data processing framework such as CMSSW. We also compare the achieved event processing throughput to the original CUDA code and a CPU version of it.
August 2021
Link to Paper
Toward Real-time Analysis of Experimental Science Workloads on Geographically Distributed Supercomputers
While next-generation exascale supercomputers promise strong support for I/O-intensive workflows, HPC remains largely untapped by live experiments, because data transfers and disparate batch-queueing policies are prohibitive when faced with scarce instrument time. To bridge this divide, we introduce Balsam
June 2021
Link to Paper
While next-generation exascale supercomputers promise strong support for I/O-intensive workflows, HPC remains largely untapped by live experiments, because data transfers and disparate batch-queueing policies are prohibitive when faced with scarce instrument time. To bridge this divide, we introduce Balsam
June 2021
Link to Paper
Deep Reinforcement Agent for Scheduling in HPC
HPC scheduling agent named DRAS (Deep Reinforcement Agent for Scheduling) by leveraging deep reinforcement learning. DRAS is built on a novel, hierarchical neural network incorporating special HPC scheduling features such as resource reservation and backfilling.
April 2021
Link to Paper
HPC scheduling agent named DRAS (Deep Reinforcement Agent for Scheduling) by leveraging deep reinforcement learning. DRAS is built on a novel, hierarchical neural network incorporating special HPC scheduling features such as resource reservation and backfilling.
April 2021
Link to Paper
Physics Object Localization with Point Cloud Segmentation Networks
Performing semantic segmentation on low-level ATLAS detector simulation using Point Cloud and Graph Neural Networks.
March 2021
Link to Paper
Link to Public Dataset
Performing semantic segmentation on low-level ATLAS detector simulation using Point Cloud and Graph Neural Networks.
March 2021
Link to Paper
Link to Public Dataset
HL-LHC Computing Review Stage-2, Common Software Projects: Event Generators
This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group (WG), as an input to the second phase of the LHCC review of High-Luminosity LHC (HL-LHC) computing, which is due to take place in November 2021.
September 2021
Link to Paper
This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group (WG), as an input to the second phase of the LHCC review of High-Luminosity LHC (HL-LHC) computing, which is due to take place in November 2021.
September 2021
Link to Paper
|
The Department of Energy: AI For Science ReportReport that envisions the DOE Artificial Intelligence research landscape for the next 10-20 years.
March 2020 Link to Report |
Balsam: Near Real-Time Experimental Data Analysis on SupercomputersSupercomputing 2019: Workshop on Large-scale Experiment-in-the-Loop Computing (XLOOP)
Link to Paper |
|
|
Balsam: Automated Scheduling and Execution of Dynamic, Data-Intensive HPC Workflows |
Building and Using Containers on HPCs for the ATLAS ExperimentIntl. Conf. on Computing in High Energy and Nuclear Physics (CHEP)
July 2018, Sofia, Bulgaria Link to Paper |
|
|
An Edge Service for Managing HPC Workflows |
Challenges in scaling NLO generators to leadership computers
Journal of Physics: Conference Series (Volueme 898)
22nd International Conference on Computing in High Energy and Nuclear Physics (CHEP2016)
10–14 October 2016, San Francisco, USA
Link to Paper
Journal of Physics: Conference Series (Volueme 898)
22nd International Conference on Computing in High Energy and Nuclear Physics (CHEP2016)
10–14 October 2016, San Francisco, USA
Link to Paper
Precision searches in dijets at the HL-LHC and HE-LHC
Journal of Instrumentation (Volume 13)
May 2018
Link to Paper
Journal of Instrumentation (Volume 13)
May 2018
Link to Paper
Developments in Architectures and Services for using High Performance Computing in Energy Frontier Experiments
Proceedings of Science (Volume 178)
38th International Conference on High Energy Physics
3-10 August 2016, Chicago, USA
Link to Paper
Proceedings of Science (Volume 178)
38th International Conference on High Energy Physics
3-10 August 2016, Chicago, USA
Link to Paper
Adapting the serial Alpgen parton-interaction generator to simulate LHC collisions on millions of parallel threads
Computer Physics Communications
Volume 210, January 2017
Link to Paper
Computer Physics Communications
Volume 210, January 2017
Link to Paper
Simulation of LHC events on a millions threads
Journal of Physics: Conference Series (Volume 664)
21st International Conference on Computing in High Energy and Nuclear Physics (CHEP2015)
13-17 April 2015, Okinawa, Japan
Link to Paper
Journal of Physics: Conference Series (Volume 664)
21st International Conference on Computing in High Energy and Nuclear Physics (CHEP2015)
13-17 April 2015, Okinawa, Japan
Link to Paper
Achieving production-level use of HEP software at the Argonne Leadership Computing Facility
Journal of Physics: Conference Series (Volume 664)
21st International Conference on Computing in High Energy and Nuclear Physics (CHEP2015)
13-17 April 2015, Okinawa, Japan
Link to Paper
Journal of Physics: Conference Series (Volume 664)
21st International Conference on Computing in High Energy and Nuclear Physics (CHEP2015)
13-17 April 2015, Okinawa, Japan
Link to Paper
Measurements of normalized differential cross-sections for ttbar production in pp collisions at sqrt(s) = 7TeV using the ATLAS detector
Phys. Rev. D 90, 072004
Published 13 October 2014
Link to Paper
Phys. Rev. D 90, 072004
Published 13 October 2014
Link to Paper
- “ATLAS Level-1 Calorimeter Trigger: PreProcessor Implementation and Performance,” J.T. Childers, et al., JInst. 7 P12008, 2012
- “Performance of the Electron and Photon Trigger in p-p Collisions at sqrt(s) = 7 TeV,” J.T. Childers and A. Tricoli (for ATLAS Collaboration), ATLAS-CONF-2011-114, 2011
- “ATLAS level-1 calorimeter trigger hardware: initial timing and energy calibration,” J.T. Childers (for ATLAS Collaboration), J. Phys.: Conf. Ser. 293, 2011
- “ATLAS Level-1 Calorimeter Trigger: Timing Calibration with 2009 LHC Beam Splashes,” J.T. Childers (for ATLAS Collaboration), ATL-DAQ-PUB-2010-001, 2010
- “Expected boron to carbon at TeV energies,” J.T. Childers and M.A. DuVernois, Proc. 30th Int. Cosmic Ray Conf., Merida, Vol. 2, pgs. 183-186, 2008
- “Timing charge and position analysis from the first CREAM flight,” J.T. Childers, Proc. 30th Int. Cosmic Ray Conf., Merida, Vol. 2, pgs. 373-376, 2008
