This blog continues our series called “AI Essentials.” It discusses the role of data centers in AI development, the questions being raised by stakeholders, and what this means for competition and innovation in the AI ecosystem.
If data and compute are integral to AI innovation, then the facilities that house those resources play an obvious central role and are essential to understand as well. You can think of a data center as akin to a factory, where raw materials (data) are processed using machines (compute) to create a new output. The growth of AI innovation has increased demand for these “factories,” leading to renewed policy conversations about energy, natural resources, and competitiveness.
Data centers have been around for decades and are physical locations that house the computing infrastructure — servers, storage and networking equipment — that underlie the services and applications we all use. Traditional data centers power Internet applications — from the mundane social media scrolling you were doing to happen across this blog post, to the critical advanced medical imaging machines helping save lives in the hospital across town. These data centers are primarily comprised of central processing unit (CPU)-based servers. Meanwhile, AI-focused data centers house servers with graphics processing units (GPUs) or even tensor processing units (TPUs). (As we explained in an earlier post from this series, CPUs process information sequentially whereas GPUs perform calculations in parallel, making them excel at AI-related tasks.)
AI data centers are essential for both model development and training as well as for running finished AI applications. Startups — and even some leading model developers — often rely on cloud infrastructure to train (or fine-tune) and deploy their AI services. Large model developers often have their own dedicated infrastructure to train and deploy their models. Many leading technology companies are planning to build new AI data centers to keep up with advancing technological developments and increasing demand.
Like any “factory,” data centers, of course, require energy to operate. In addition to powering the servers and networking equipment, part of data centers’ energy use (perhaps up to half) is the need to be cooled. Server equipment generates heat and needs to stay within a specific temperature and humidity range (between ~64 and ~80 degrees fahrenheit and 40 to 60 percent relative humidity) to prevent declines in performance or damage to the equipment.
There are many air and liquid cooling technologies deployed in data centers, which is critical for efficiency and proper facility performance. Often, AI data centers’ more powerful chips must be liquid-cooled. Liquid cooling system designs vary depending on the facility. They typically use reclaimed wastewater (not potable water used for drinking), but some may additionally draw on other freshwater sources. Increasingly, facilities are closed-loop and recirculate the same fluid. Because of the essential nature of cooling for performance, data centers need backup generators (usually diesel powered) in the event of a power failure. (Notably, air cooled facilities can manage a power lapse a bit longer as ambient air temperature slowly rises in the absence of cooling, whereas liquid-cooled facilities lose effectiveness as soon as flow of the liquid stops).
The energy consumption of data centers have led some policymakers and environmental activists to introduce legislation regulating the construction or resource use of AI data centers and protesting their construction or connection to the grid. The impact on cost per kilowatt-hour due to increasing demand outpacing supply is a central focus of the complaints. But it is important to note that while energy use has increased, facility efficiency has increased faster, meaning the energy intensity of data centers has decreased by 20% annually. Moreover, industry has explored methods to increase supply of energy, like co-locating data centers with power plants and bringing additional power generation facilities online.
However, some have proposed making data center operators pay for utility upgrades in addition to paying for power they use. (That model has been regularly proposed in the telecommunications space, and effectively amounts to paying twice). Those costs are likely to be passed on to those using the data center. Sometimes that will be a large model developer, who will then disperse those costs to end users, like startups who fine-tune the models to create unique services. Sometimes that will be startups directly. Moreover, the earliest stage startups often benefit from computing credits that enable them to get off the ground with few resources. Increasing costs for startups would be harmful and eat into startups’ limited resources, undermining their ability to innovate with AI and the competitiveness of the broader AI ecosystem.
