Just try solving the mysteries of the universe without high performance computing in your tool box.

Certainly, the physicists, many of them at Purdue, involved in the CMS experiment connected to the Large Hadron Collider wouldn’t do it. The LHC is the circular, 17-mile-long particle accelerator starting up in earnest in France and Switzerland this fall and set to be officially commissioned in October. Yes, it’s big enough to be in parts of two countries.

“It’s by far the largest and most complicated scientific instrument ever built by mankind,” Purdue physics professor Norbert Neumeister said of the collider, which is designed to crash high-energy protons into each other nearly a billion times per second at almost light speed, generating even more elementary particles in the process and recreating conditions like those shortly after the universe-forming Big Bang.

Currently, several Purdue research scientists and students are working on site at the Switzerland-based European Organization for Nuclear Research, known as CERN, the particle physics laboratory managing the accelerator for an international consortium that takes in thousands of physicists at more than 100 research institutions in more than two dozen countries. The facility took 14 years and $8 billion to build.

Also on site from Purdue: a key component of the LHC detectors looking for elementary particles that could explain questions yet unanswered about particle physics’ Standard Model or, perhaps, modify our understanding of the way space, time and other important things work.

“It can really change the perspective of what’s going on in nature,” said Neumeister, who was at CERN this summer.

Meanwhile, headed in the direction of Purdue and Information Technology at Purdue’s Rosen Center for Advanced Computing will be reams of data from the proton collisions in the accelerator, to be assembled into a virtual library used by physicists on campus and the world over.

A CMS Tier-2 Center at Purdue, with Neumeister as principal investigator, is built around high performance computing systems administered and supported by the Rosen Center. The resources will support the computational needs of physicists involved in the project in the U.S and abroad.

CMS stands for Compact Muon Solenoid, one of four big experiments associated with the Large Hadron Collider. It’s one of the two general purpose LHC experiments aimed at a region where scientists believe the major questions in particle physics still pending in the 21st Century could be answered, or new questions altogether yielded.

The detector is to pick up on muons and other particles unleashed by the collisions generated in the system, perhaps even the as-yet-unseen Higgs boson. The Higgs could explain how massless particles combine to create mass in matter and confirm, or open new vistas on, the Standard Model even if it doesn’t force the model’s rethinking.

The Standard Model is currently the best explanation of the fundamental interactions among the 12 elementary particles that make up matter and the four forces which prod them to interact. But it has shortcomings, an explanation of mass for one.

The LHC and its detectors also may generate new insight into dark matter, black holes, the possibility of extra dimensions, and more. The collider’s four detectors will count, trace and analyze the particles that pass by, recording enough data annually to fill a stack of CDs 12 miles high.

The massive amount of data from the CMS detector alone is being handled by three levels of centers, starting with the Tier-0 near the detector, which collects and pre-processes the data, creates a full backup and parcels the collection out to a handful of Tier-1 centers for further processing and refining, including Purdue’s partner Fermi National Accelerator Laboratory near Chicago.

As a Tier-2 center, Purdue will do quality monitoring, analysis and archiving of data, the latter in a manner that offers physicists convenient and easy access to it. Tier-2 centers basically hold the chapters of the full data “book” in which their designated users have the most interest.

In Purdue’s case, that includes physicists interested in muons and heavy electrons and in “exotic physics,” which is to say particles and phenomena we know nothing about but which may be revealed by the powerful Large Hadron Collider.

Via the Open Science Grid, those physicists will tap the resources at Purdue’s Tier-2 Center and the Rosen Center to work with the data, Neumeister said.

“It could well be a guy in Russia or a guy in China is submitting jobs to our Tier-2 center,” he said.

Neumeister said the work of the Tier-2 centers, of which there are seven in the U.S., also involves large amounts of simulation to compare the data with physics theories. That likewise requires the high performance computing resources available through the Rosen Center.

The CMS Tier-2 Center at Purdue was a major investor in the Rosen Center’s new 848-node, 60-teraflop Steele cluster, built in May and funded by contributions from more than two dozen Purdue faculty members and research organizations. Neumeister said the center also will be a major investor in Rosen’s next round of “community clustering,” in the planning stages now for the spring of 2009.

At times, the Tier-2 center uses up 100 percent of its dedicated resources, Neumeister said, but also can draw on the computing nodes of other users who may not be occupying them at the time, one of the ideas behind community clusters.

Purdue also has dedicated more than 500 terabytes of storage to the effort.

As the Sept. 10 initial startup of the Large Hadron Collider approached, Purdue researchers were running hefty test jobs using simulated data and sharing results back and forth with Fermilab, just as will be required when the collisions and work with real data begin later this fall and early next year in a gradual ramping up of the accelerator.

The particular data around which the Purdue center will revolve is a likely place for mystery-solving revelations, said Neumeister and Purdue physics Professor Daniela Bortoletto, whose research focuses on the Higgs and the possibility of particles not predicted by the Standard Model.

“For me it’s really a dream,” Bortoletto said. “We think that the (Large Hadron Collider) will finally give us the opportunity of finding these particles if they exist. It’s like turning the lights on. I really expect a lot of discovery.”

While Bortoletto’s ultimate focus is on new physics, she has a double stake in the startup of the Large Hadron. Her Purdue lab in collaboration with Purdue Professor Ian Shipsey’s lab and other Purdue colleagues designed and helped build, in partnership with Fermilab and European researchers, what’s really the detecting part of the CMS detector.

Immersed in a 14,500-ton structure the size of a six-story office building, the Purdue-developed silicon pixel detector is by comparison a small—it weighs a couple pounds and you could hold it in your hands—but vital component. It works like a digital camera, only capturing evidence of particles from the collisions in the accelerator instead of the photons a camera captures to make pictures, and at a rate of more than 60 million pixels instead of the few million in even the best digital cameras.

Bortoletto and Shipsey began collaborating on the device after she joined the CMS project and searched for someone to build a silicon pixel detector to her design. Shipsey already had gained a reputation building similar devices, including a detector for CLEO III, a project looking at quark particles created when matter and antimatter collide. The CMS detector has been in development since 1996.

“It’s essential,” said Bortoletto, who also is coordinating an upgrade already planned for the detector. “It’s a tiny but very important component of CMS.”

Important enough that two Purdue senior scientists, Gino Bolla and Petra Merkel, were dispatched to CERN in May to help guide its installation and commissioning and will be there until at least next spring.

Bolla, who by August hadn’t had a weekend off for five weeks, is the contact point between the pixel detector group and the rest of CMS operations, while Merkel leads a software group responsible for data quality monitoring. Both were working long hours during the day as September approached, and working at home or returning evenings.

“Things are very hectic,” Merkel wrote in an e-mail. “Our detector is working great, but the system as a whole is extremely complex and every day there seems to be something else broken.” Still, she said the dominant feeling on site was excitement.

“People are VERY excited to see this giant project finally taking off, after so many years of waiting,” she wrote.

Shipsey thinks the excitement extends to particle physics as a whole, rejuvenating the field. He’s noticed an increase in student interest for one thing, particularly as media coverage of the collider’s kickoff has mounted.

“I’ve never seen anything this exciting,” he said.

Purdue physicists working on the LHC also include professors Virgil Barnes, Art Garfinkel, Laszlo Gutay, Matthew Jones and David Miller. The Purdue Physics Department, Miller and Bortoletto have prepared a Web site about the LHC and CMS aimed especially at high school students and teachers, although likely of interest to the general public as well. The site is at: www.physics.purdue.edu/particle/lhcf.

Writer: Greg Kline, (765) 494-8167, gkline@purdue.edu

Sources: Norbert Neumeister, (765) 494-5198, neumeist@purdue.edu
Daniela Bortoletto, (765) 494-5197, bortolet@purdue.edu
Ian Shipsey, (765) 494-5391, shipsey@purdue.edu
Petra Merkel, petra.merkel@cern.ch

More information: www.physics.purdue.edu/particle
www.physics.purdue.edu/Tier2
www.rcac.purdue.edu

Last updated: Sept. 18, 2008

Purdue University, West Lafayette, IN 47907 USA, (765) 494-4600
© 2009 Purdue University. An equal access, equal opportunity university.

Inquiries or comments: itap@purdue.edu