How the massive energy consumption of generative AI is straining our grid

Thanks to the AI ​​boom, new data centers are being built as quickly as companies are building them. This has translated into massive demand for energy to power and cool the servers inside them. Now concerns are growing about whether the United States can generate enough electricity to support widespread adoption of AI, and whether our aging grid will be able to handle the load.

“If we don’t start thinking about this power problem differently now, we’ll never see this dream we’re dreaming of,” said Deepti Vachani, head of automotive at Arm. The chipmaker’s low-power processors are becoming increasingly popular with large companies like Google, Microsoft oracle and Amazon – Specifically because it is able to reduce energy usage by up to 15% in data centers.

NvidiaAmazon's latest AI chip, Grace Blackwell, includes Arm-based CPUs and says it can run generative AI models using about 25 times less power than the previous generation.

“Save every last drop of power would be a completely different design than one where you’re trying to maximize performance,” Vachani said.

This strategy of reducing energy use by improving computing efficiency, often referred to as “more work per watt,” is one answer to the AI ​​energy crisis. But it’s not nearly enough.

According to a report from Goldman Sachs, a single ChatGPT query consumes nearly 10 times the energy of a regular Google search. And generating an image with AI can consume as much energy as charging your smartphone.

This problem is not new. A 2019 study found that training a large language model produced as much carbon dioxide as five gas-powered cars over their lifetime.

The massive companies that build data centers to accommodate this massive energy consumption are also seeing their emissions rise. Google’s latest environmental report showed that its greenhouse gas emissions rose by about 50% from 2019 to 2023 in part due to data center energy consumption, though it also said its data centers are 1.8 times more energy efficient than a typical data center. Microsoft’s emissions rose by about 30% from 2020 to 2024, also in part due to its data centers.

And in Kansas City, where Meta is building an AI-focused data center, energy needs have become so high that plans to close a coal-fired power plant have been put on hold.

There are more than 8,000 data centers worldwide, with the highest concentration in the United States, and thanks to AI, there will be more by the end of the decade. The Boston Consulting Group estimates that demand for data centers will grow 15%-20% each year through 2030, when they are expected to account for 16% of total U.S. energy consumption. That’s up from just 2.5% before OpenAI’s ChatGPT launched in 2022, and is equivalent to the energy used by about two-thirds of all U.S. homes.

CNBC visited a Silicon Valley data center to see how the industry is handling this rapid growth, and where it will find the muscle to make it possible.

“We believe the demand we will see from AI applications will be as high or higher than what we have historically seen from cloud computing,” said Jeff Tench, executive vice president of Vantage Data Center, North America and Asia Pacific.

Many major tech companies contract with companies like Vantage to host their servers. Vantage’s data centers typically have the capacity to use up to 64 megawatts of power, or the equivalent of tens of thousands of homes, Tench said.

“Many of these units are being used by individual customers, who will be renting out the entire space to them,” Tench said. “When you think about AI applications, these numbers can grow dramatically into the hundreds of megawatts.”

Santa Clara, California, where CNBC visited the Vantage site, has long been one of the hottest places in the country to build data centers near data-hungry customers. Nvidia’s headquarters was visible from the rooftop. There’s been a “slowdown” in Northern California because of “the lack of power availability from the utilities here in this area,” Tench said.

Vantage is building new locations in Ohio, Texas and Georgia.

“The industry itself is looking for places where there is direct access to renewables, whether it’s wind or solar, and other infrastructure that can be leveraged, whether it’s as part of an incentive program to convert what would have been a coal-fired plant to natural gas, or increasingly looking for ways to harness power from nuclear facilities,” Tench said.

The aging electrical grid is often ill-equipped to handle the load even when enough power can be generated. The bottleneck is in moving power from where it is generated to where it is consumed. One solution is to add hundreds or thousands of miles of transmission lines.

“This is very expensive and time-consuming, and sometimes the cost is passed on to residents through increased utility bills,” said Shaoli Ren, an assistant professor of electrical and computer engineering at the University of California, Riverside.

A $5.2 billion effort to extend lines into an area of ​​Virginia known as “Data Center Alley” has met with opposition from local taxpayers who don’t want to see their bills increase to fund the project.

Another solution is to use predictive software to reduce faults at one of the network's weakest points: the switch.

“All the electricity generated has to go through a transformer,” said VIE Technologies CEO Rahul Chaturvedi, adding that there are 60 million to 80 million of them in the United States.

Transformers also have an average lifespan of 38 years, making them a common cause of power outages. Replacing them is expensive and slow. VIE makes a small sensor that attaches to transformers to predict failures and determine which transformers can handle more load so it can be diverted away from those at risk of failure.

Chaturvedi said business has tripled since ChatGPT launched in 2022, and is expected to double or triple again next year.

According to Rain’s research, generative AI data centers will also need to withdraw between 4.2 billion and 6.6 billion cubic meters of water by 2027 to stay cool. That’s more than the total annual water withdrawal of half the UK.

“Everyone is worried about AI’s energy consumption,” said Tom Ferguson, managing partner at Burnt Island Ventures. “We can solve that problem when we move on and stop being so silly with nuclear power, right? That’s solvable. Water is the fundamental determinant of what’s to come with AI.”

Ren's research team found that every 10 to 50 ChatGPT prompts could burn off what you'd find in a standard 16-ounce water bottle.

Much of that water is used for evaporative cooling, but Vantage's Santa Clara data center has large air conditioning units that cool the building without having to draw any water.

Another solution is to use liquid to directly cool the chip.

“For many data centers, this requires a massive amount of modification and upgrade. In our case at Vantage, about six years ago, we deployed a design that allowed us to take advantage of a cold water loop here on the floor of the data hall,” said Vantage’s Tench.

Companies like appleSamsung and Qualcomm They touted the benefits of on-device AI, which keeps power-hungry queries out of the cloud, and away from power-starved data centers.

“We’re going to get as much AI as these data centers support,” Tench said. “That may be less than people are hoping for. But ultimately, there are a lot of people working on ways to mitigate some of these supply constraints.”