November 9, 2023

AI refers to the ability of machines and computer systems to imitate intelligent human behavior, learn from experience, and make adjustments based on new information. While we can easily interface with and use AI, it may be difficult to understand how it works. There is a behind-the-scenes network of data centers that keeps many industries, items, and ideas running. The Internet of Things (IoT), cloud computing, and search engines rely on these data centers to operate. So does AI, and like emergent technologies in the past, it is impacting data center infrastructure. Data centers have numerous components (shelves, cabinets, and enclosures don’t even scratch the surface of what makes up a data center), and every piece of custom-designed equipment needs to function together to make the data center operational. Contractors and companies that produce AI technology often depend on experienced manufacturers of server racks, switches, cooling systems, and various components to build AI data center enclosures. AI is rapidly changing the physical structure of data centers from the inside out as industries adapt to incorporate artificial intelligence and machine learning.

The Significance of AI Today

To promote AI innovation, establish safeguards, and protect Americans’ privacy, the White House issued an Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence this month. The order also aims to modernize federal AI infrastructure and to ensure the responsible government deployment of AI.

While government agencies and major businesses utilize AI, so do everyday citizens. Chatbots write the blogs you read (not this one), vehicles are equipped with object detection and accident prevention capabilities, and digital services suggest news and entertainment based on your interests. LinkedIn recently unveiled a new AI-powered chatbot to help users find jobs. It is estimated that generative AI could raise the global GDP by 7% (by comparison, innovations in electricity and personal computing raised the GDP by only 2%). By 2025, investments in these advancements could reach nearly $100 billion in the United States and more than $200 billion worldwide, according to a Goldman Sachs report. Artificial intelligence is anything but artificial. It’s real, and it’s everywhere.

The Recent Emergence of Large Language Models

When you interact with different AI-powered products and services, a large language model (LLM) is the algorithm that takes the natural language input and generates human-like responses or actions. In other words, an LLM is what makes it so that you don’t have to speak “computer” to talk with an AI tool. Over the last year, people have become more aware of LLMs because of the mainstream emergence of open-source models, like Chat GPT and Google’s LaMDA and PaLM.

LLMs exist in a smaller data center close to a larger data center, referred to as a hyperscale data center. LLMs use the large amounts of data being stored in the hyperscale data center to train themselves. LLMs create more data and content than a typical data center, and they have more demanding compute and cooling needs than a traditional data center.

Data Center Infrastructure: Data, Network, and Compute

Data is created, stored, and consumed at a rapid pace these days, and it is the fuel for the Information Communication Technology industry. A network is the avenue in which data moves in between devices and storage systems. Compute refers to the force that organizes and processes data. The growth, development, and use of AI has had a major impact on how data is processed, stored, and used. In fact, with the artificial intelligence boom this year, the cost of data center space has increased and so has the power consumption requirements.

Data centers are communication hubs and the arteries of the internet. They contain infrastructure for power supply, data communication connectivity, security devices, and environmental controls. Housed inside are routers and switches that move data, communication, and energy between servers and the outside world. Data consumption has increased exponentially as AI advancements have been adopted, and it is projected to continue that upward trend in years to come.

With the introduction of AI, data centers are adapting to the increasing demands and complexity of AI algorithms and neural networks (a branch of machine learning that mimics the human brain). Specialized hardware is necessary to support this neural network and allow data centers and computers to process information in this way. Manufacturers like Cadrex partner with the companies leading the AI data center charge to produce custom server racks, server enclosures, and cooling systems that make up modern data center infrastructure.

Data Center Infrastructure for AI

The rise of AI is changing the landscape of data center infrastructure. To incorporate AI, data centers must account for higher-density workloads, powerful (and hot) technology, and additional equipment taking up space. As AI software advances , so do the hardware and mechanical technologies needed to solve the challenges it puts upon data centers.

Specialized Hardware

Accommodating new technology comes with hardware challenges in AI data centers. In the past, data centers used central processing units (CPUs) to execute computer programming instructions. Hardware requirements for AI workloads necessitate the use of graphics processing units (GPUs) instead. GPUs were initially designed to accelerate computer graphics and image processing. Now, GPUs are designed with AI workload enhancements and are optimized for algorithm training and deep machine learning. GPU servers for AI consume more power than CPUs. Physically, they are larger and hotter than CPUs, so they require larger racks and advanced cooling systems to keep them functioning and to protect other hardware. These structural requirements are the primary drivers behind so many AI and machine learning companies choosing to build custom mechanical solutions in their data centers.

Keeping it Cool

Data center cooling for AI is a challenge that stems from the use of GPUs. They are incredibly powerful, and they emit a great deal of heat. These changes in hardware for AI servers require alternative cooling systems that must cool data centers efficiently without wasting money or energy. Traditionally, data center operators may have relied on cooler air circulation as the main method to keep hardware cool. Modern cooling methods include using fewer racks per row, reorganizing the geometrical design of racks in data centers, and having smaller rooms. New methods include liquid cooling systems or immersion cooling, which involve submerging equipment in liquid coolant. These cooling techniques require less water than air-cooling systems and reduce the carbon footprint of a data center.

AI Workload Optimization

The data center transformation for AI involves establishing and meeting hardware requirements for AI workloads. Existing data centers can be fitted with AI-specific servers to accommodate a surge in workloads across data center resources. New data centers are designed around custom-built server racks, cooling systems, and more energy efficient power infrastructure that optimize AI workloads.

Looking Towards the Future

Artificial intelligence will only get smarter from here, and new technologies to incorporate its future are on the horizon. It is predicted to become even more prevalent in our daily lives, and it will reshape and revolutionize industries, companies, and products all over the world. Many predict generative AI, which uses deep learning to create new content, is a phase that will make way for interactive AI, which is the use of AI to carry out more day-to-day tasks.

About Cadrex

At Cadrex, we have decades of experience building the niche mechanicals that make up the physical infrastructure of data centers. We use sheet metal fabrication, precision machining, plastic injection molding, and state-of-the-art technology to help innovative companies build the next generation of technology. Learn more about our ICT market expertise and ICT manufacturing at Cadrex.