Search for Higgs boson pair production in the two bottom quarks plus two photons final state in collisions at with the ATLAS detector

Searches are performed for nonresonant and resonant di-Higgs boson production in the b (b) over bar gamma gamma final state. The dataset used corresponds to an integrated luminosity of 139 fb(-1) of proton-proton collisions at a center-of- mass energy of 13 TeV recorded by the ATLAS detector at the CERN Large Hadron Collider. No excess above the expected background is found and upper limits on the di-Higgs boson production cross sections are set. A 95% confidence-level upper limit of 4.2 times the cross section predicted by the Standard Model is set on pp -> HH nonresonant production, where the expected limit is 5.7 times the Standard Model predicted value. The expected constraints are obtained for a background hypothesis excluding pp -> HH production. The observed (expected) constraints on the Higgs boson trilinear coupling modifier kappa(lambda) are determined to be [-1.5, 6.7] ([-2.4, 7.7]) at 95% confidence level, where the expected constraints on kappa(lambda) are obtained excluding pp -> HH production from the background hypothesis. For resonant production of a new hypothetical scalar particle X (X -> HH -> b (b) over bar gamma gamma), limits on the cross section for pp -> X -> HH are presented in the narrow-width approximation as a function of m x in the range 251 GeV <= m(X) <= 1000 GeV. The observed (expected) limits on the cross section for pp ->> X -> HH range from 640 fb to 44 fb (391 fb to 46 fb) over the considered mass range.