Big tech needs to end LLM arms race, ditch carbon backpacks, says IBM’s former AI chief

Tools like ChatGPT run on LLM, artificial neural networks that have been trained on vast amounts of data scraped from the web and provide AI-generated answers to text-based prompts.

Speaking at Fortune’s Brainstorm AI conference in Singapore last week, Seth Dobrin said the future may belong to small language models (SLMs), which are tailored to solve specific applications and require less energy to run. Much less.

“These huge, large models are not what this technology is about. They’re cool, they’re interesting, but they’re not the solution,” Dobrin, general partner at venture capital fund 1infinity Ventures, told attendees last week. “Consider using small, mission-specific models.”

Experts have long warned that artificial intelligence will not reach its full potential until its addiction to energy is addressed. Arm Holdings, a company that designs energy-saving microchips for handheld devices, predicted earlier this year that GenAI could consume a quarter of all U.S. electricity by 2030.

It’s not just energy, either. Many data centers also use water and air to cool servers as they process terabytes of data in seconds.

Dobrin said that few people today are aware of the ecological impact of using ChatGPT or Claude.

“For every 25 to 50 tips, depending on the size of the tip, you use about half a liter of water – just by evaporation,” he says. “We absolutely need a new cooling paradigm.”

Unfortunately, as miniaturization advances and process technologies shrink to the 2 and 3 nm nodes (the smallest size circuits that can be printed onto silicon wafers), their thermal performance is becoming an increasing issue.

This creates a growing dilemma for the industry, as fans and air conditioners cannot move heat away fast enough, and even cooling plates attached directly to the wafers fail to function beyond a certain computing speed.

Immerse the server in an oil bath to quickly dissipate heat

Tim Rosenfield, co-founder and co-CEO of Sustainable Metal, said: “We are becoming less and less efficient at every step, and this coincides with the rise of artificial intelligence, which unfortunately uses the most energy-intensive and Popular chips.

He believes SMC may have the answer. Its flexible, modular HyperCube data center hardware can substantially reduce the carbon footprint of traditional air-cooled H100 HGX systems by immersing servers directly in an oil bath, a relatively new process called immersion cooling. half.

Oil is more efficient at extracting heat than air and is less conductive than water, making it better suited for server racks that must run cutting-edge 2nm and 3nm AI training and inference chips.

“The problem we’re trying to solve—how to convert energy into knowledge as cost-effectively as possible—starts with heat,” Rosenfeld said. “This is one of the drivers of the energy challenge for artificial intelligence.”

While immersing the servos directly in an insulating liquid helps meet the need to dissipate growing heat quickly and efficiently, the new technology also comes with challenges in terms of upfront investment costs, maintenance and repairs.​​​

So Rosenfeld’s plan is to offer it as a complete package deal, which he calls “cooling as a service.”

Venture capitalist Doblin also has an unconventional suggestion that could help minimize artificial intelligence’s massive carbon backpack, in addition to using new cooling technologies.

Before using a powerful tool like ChatGPT or Claude, forget the GenAI hype for a moment and first ask yourself if there are other tools that are equally suitable for the task.

“Focus on the use case—what problem are you trying to solve?” he said. “Do you really need generative artificial intelligence?”

CEO Daily provides the critical context news leaders in the business world need to know. Every weekday morning, more than 125,000 readers trust CEO Daily to get insights on the C-suite and beyond. Subscribe now.