Any firm that wants a competitive edge in computing must bring together the best of its information technology (IT), high-performance computing (HPC) and artificial intelligence (AI) expertise. That does not mean a negotiated co-existence between them—it means developing true enthusiasm and cooperation.
Automotive, consumer products, pharmaceutical, heavy industry and other sectors are using HPC and adding activities in AI. However, companies often have two divides: HPC vs. IT, and HPC vs. AI. It is disappointing how much miscommunication can occur due to these divides. The opportunity for companies to bridge these divides is substantial, including the ability to beat external competition by reducing internal competition across these divides.
Many people tell me that HPC is not IT! While sharing this as a key insight he sees in the best managed HPC organizations, Andrew Jones of NAG (a non-profit with expertise in numerical engineering) also observed that “HPC is built using IT components, but so are radio telescopes, etc.
His point is that we must not let the similarities between HPC and IT (both use computers, right?) mask what should be different! A fundamental difference is that IT is generally operated as an accepted cost, whereas HPC is a value generator (and should be managed as such).
Comprehending this may be especially difficult with those who sold management on the value of high performance computing (HPC) once, but then struggle to make it a permanent part of a company’s culture. Equating HPC to IT will not lead to the best results. I’m not advocating for a great divide. Cooperation and understanding are highly encouraged, but the goals against which HPC is managed need to be fundamentally different.
In most companies, expertise in HPC and AI are aligned with different decision makers who find themselves in internal competition instead of aligned in discovering possibilities. However, there is huge opportunity here, and companies that bring these factions together successfully are in the best position to beat their competition!
An example of AI in manufacturing is the rise of autonomous mining vehicles, such as those produced by Caterpillar. While HPC experts use modeling and simulations to work out design mechanics, AI experts imagine the possibilities of increased safety and utilization in a design.
Constant Engagement Needed
A special report from insideHPC, a blog focused on HPC, discusses the results from a multi-year collaboration between Stanford University, NERSC, and Intel. The report, titled “AI-HPC is Happening Now,” noted that the result was the successful training of deep-learning neural networks with 9600 computational nodes.
Prior to this result, the literature only reported scaling of deep learning training to a few nodes. Keeping up with this technology requires engagement with experts at conferences, in user groups, and with vendor contacts. The report also discussed making HPC more approachable for “the rest of us” with initiatives like Intel Select Solutions for Simulation and Modeling.
Many will read this. Many will nod their heads. Most will do nothing. If it was easy, everyone would do it. Will you be in the minority, and make a real difference for your organization by making it a real competitive advantage?