The Surprising Scale of AI's Energy Consumption: A Deep Dive

The Rising Energy Demands of AI

The rapid advancement of Artificial Intelligence (AI) has sparked widespread debate, with its staggering energy consumption emerging as a key concern. Some jokingly suggest that AI might only fail to replace humans when electricity becomes more expensive than bread. This quip underscores a serious reality: high energy consumption could become a bottleneck in AI's development. A former Google engineer, Kyle Corbitt, revealed on social media that Microsoft encountered power challenges while training GPT-6.

To train massive AI models, Microsoft engineers are constructing InfiniBand networks to connect GPUs across different locations. The complexity arises because concentrating over 100,000 H100 chips in a single area could overwhelm the local power grid, risking a complete collapse.

Let's do a simple calculation. Nvidia's data indicates that each H100 chip has a peak power of 700W. Consequently, 100,000 chips could reach a peak power consumption of 70 million watts. Energy sector professionals point out that such enormous consumption is equivalent to the entire output of a small solar or wind power plant. Furthermore, we must also consider the energy needed for servers and cooling equipment. Concentrating all these power-consuming devices in a small area puts immense pressure on the power grid.

AI Power Consumption: Just the Tip of the Iceberg

A New Yorker article gained significant attention when it estimated that ChatGPT's daily power consumption could exceed 500,000 kilowatt-hours. Despite this, AI's current power consumption pales in comparison to that of cryptocurrencies and traditional data centers. The challenges Microsoft engineers face highlight that it's not just the energy consumption of the technology itself but also the energy required for supporting infrastructure and the power grid's capacity that are limiting AI development.

A report from the International Energy Agency (IEA) showed that in 2022, global data centers, AI, and cryptocurrencies consumed 460 TWh, about 2% of global energy consumption. The IEA projects that in a worst-case scenario, these sectors could consume 1000 TWh by 2026, which is comparable to the total electricity consumption of Japan.

It's worth noting that the energy directly invested in AI research is currently much lower than that of data centers and cryptocurrencies. Nvidia, which dominates the AI server market, supplied around 100,000 chips in 2023, with an annual power consumption of about 7.3 TWh. In contrast, cryptocurrency consumption reached 110 TWh in 2022, equivalent to the energy consumption of the entire Netherlands.

Cooling Energy: An Unavoidable Consideration

Data center energy efficiency is often measured by Power Usage Effectiveness (PUE), which is the ratio of total energy consumed to the energy consumed by IT load. The closer the PUE value is to 1, the less energy a data center wastes. A report by the Uptime Institute showed that the average PUE for large global data centers in 2020 was about 1.59. This means that for every 1 unit of electricity consumed by the IT equipment, an additional 0.59 units are consumed by supporting facilities.

A significant portion of the additional energy consumption in data centers is used for cooling systems. Research indicates that cooling systems can consume up to 40% of a data center's total energy. With continuous advancements in chip technology, the power of individual devices is increasing, leading to higher power density in data centers and more stringent cooling requirements. However, improved data center designs can significantly reduce energy waste.

PUE values vary significantly across different data centers depending on factors such as cooling systems and structural designs. The Uptime Institute report revealed that European countries have reduced their PUE to 1.46, while more than one-tenth of data centers in the Asia-Pacific region still have PUE values exceeding 2.19.

To achieve energy saving and emission reduction goals, countries worldwide are taking measures. For instance, the EU requires large data centers to install waste heat recovery equipment. The US government is investing in research on more energy-efficient semiconductors. The Chinese government has also introduced policies requiring data centers to have a PUE no higher than 1.3 from 2025 and gradually increasing the use of renewable energy, aiming for 100% by 2032.

Tech Companies' Power Consumption: Hard to Conserve, Harder to Source

With the development of cryptocurrencies and AI, the size of data centers for major technology companies is constantly expanding. According to the IEA, the US had 2,700 data centers in 2022, consuming 4% of the nation's electricity. This is expected to reach 6% by 2026. Due to increasing land scarcity on the East and West Coasts, data centers are gradually moving towards the central states, but power supplies in these areas may not meet the demand.

Some tech companies are attempting to break free from the power grid by directly purchasing power from small nuclear power plants. However, this involves complex administrative approval processes. Microsoft is experimenting with using AI to assist in the application process, while Google is using AI for workload scheduling to improve power grid efficiency and reduce carbon emissions. When controllable nuclear fusion might be viable for application remains uncertain.

Climate Change: Adding Insult to Injury

AI development requires a stable and robust power grid. However, with the increasing frequency of extreme weather events, power grids in many regions are becoming more vulnerable. Climate change is leading to more frequent extreme weather events, not only increasing power demand and burdening grids but also directly impacting power grid infrastructure. The IEA reports that the global share of hydropower fell to a 30-year low in 2023, below 40%, due to the impacts of droughts, insufficient rainfall, and early snowmelt.

Natural gas is often seen as a bridge to renewable energy, but its stability is concerning under extreme winter weather conditions. In 2021, a cold wave hit Texas, causing widespread power outages, with some residents experiencing power cuts for over 70 hours. One of the main reasons for the disaster was frozen natural gas pipelines, leading to the shutdown of natural gas power plants.

The North American Electric Reliability Corporation (NERC) predicts that between 2024 and 2028, over 3 million people in the US and Canada will face an increasing risk of power outages. To ensure energy security while also achieving energy saving and emissions reduction, many countries are viewing nuclear power as a transitional measure. At the United Nations Climate Change Conference (COP 28) in December 2023, 22 countries signed a joint statement pledging to triple nuclear power generation capacity to 2020 levels by 2050. Meanwhile, with countries like China and India vigorously promoting nuclear power construction, the IEA projects that global nuclear power generation will reach a historical high by 2025.

The IEA report emphasizes that “in the face of changing climate patterns, enhancing energy diversity, improving power grid cross-regional scheduling capabilities, and adopting more resilient power generation methods are crucial.” Ensuring power grid infrastructure is not only critical for AI technological development but also for national economies and livelihoods.