Sustainability - Green AI, power and cooling
Requirement / business driver
AI's power density and energy use are first-order design constraints - the facility's power and cooling envelope can cap how many GPUs you can deploy, and sustainability/cost pressures make efficiency a design driver, not an afterthought.
Power density
AI racks draw 40-100+ kW versus ~5-15 kW for traditional racks. Air cooling cannot keep up at that density, which forces liquid cooling and reshapes facility power distribution and floor/rack design.
Cooling options
Power availability per rack (the supply-side ceiling)
The density figure cuts both ways. The numbers above assume you can deliver tens of kW to a rack, but the more common enterprise limit is the opposite: many enterprise and colocation racks are provisioned for only ~8-15 kW, and in a colo that ceiling is typically fixed - you cannot simply draw more. A single AI server with high-TDP GPUs can pull 1-2 kW or more (an NVIDIA H100's TDP is ~350W per card, and that is before the rest of the server), so only a few such servers exhaust a conventional rack's budget. When the rack cannot power a full stack you face two unattractive options: upgrade the rack feed (often infeasible in colo or older facilities), or spread the servers across more, under-filled racks - which works but strands rack space and raises cost. Plan to the server's peak (max) draw, not its average, since GPUs spike hard under load and tripping a breaker or starving cooling is worse than buying headroom; GPUs dominate the power budget, so size around them. The practical move is to confirm the per-rack power-and-cooling envelope first, then place GPU servers to that envelope - power availability, not floor space, is frequently the real limit on GPUs-per-rack.
The AI POD's worked layout puts numbers on that envelope: an air-cooled 8-GPU node draws ~12.5 kW, so a 30 kW / 42RU rack carries exactly two - a row of such racks per scale unit - while the row's middle rack collects spines, leaves, and the management cluster at ~20 kW / ~23RU, leaving ~10 kW and ~17RU for storage. Two consequences are design rather than logistics: with FE leaves centralised, a second row's front-end cabling crosses rows, so a smaller FE leaf pair per row keeps runs row-local; and because scaling past 512 GPUs is the re-cable/redesign inflection (Validated reference design - Cisco AI-ML lossless fabric (CVD)), the rack and power plan is drawn to the eventual cluster on day one, not the starter unit.
Cooling transition path (density bands)
Cooling scales as a ladder, not a swap, and the bands matter in scenarios. Air tops out around 8-16 kW/rack and cannot chase localized hot spots. The brownfield first step is the rear-door heat exchanger (RDHx) at roughly 20-40 kW/rack: a locationally deployed liquid loop fed through a CDU (coolant distribution unit) that solves spot problems without changing the hall's layout - and doubles as the on-ramp that brings facility water to the row for what follows. Direct-to-chip (DtC) cold plates serve the ~150 kW/rack class, scaling through liquid-to-liquid CDUs on a building loop; single-phase immersion reaches about 250 kW/rack; and immersion combined with DtC covers 350+ kW/rack extreme-density islands. Two design points ride on the ladder. The tiers are compatible and coexist - the transition is incremental and situational, starting from the installed base rather than a forklift. And the rack-density target chooses the tier (Scale-up vs scale-out sets the density), while the tier back-constrains floor loading, plumbing, and how much of the heat ends up in liquid loops where it can be recovered.
Green AI / efficiency
- Utilization is sustainability: idle GPUs waste power and money, so keeping GPUs busy (minimizing stragglers, Latency and the straggler problem) is also an efficiency and cost win - this is "effective use of accelerators."
- PUE, power capping, heat reuse, and renewable siting improve the energy profile.
- Affordability / CPU-GPU tweaking - right-size and tune so you don't over-provision; optics power (Optics and cabling selection) and DPU offload (AI-enabling hardware - GPU, DPU, SmartNIC) reduce the energy per useful unit of work.
Two numbers sharpen these levers. Facility power splits roughly 43% servers, 43% cooling and power provisioning (chillers, fans and pumps, UPS and transformer losses), 11% storage, and only ~3% network - and savings cascade: every watt saved at the compute load saves ~1.55 W at the facility, because the load's cooling and distribution overhead scales away with it. Load-side efficiency (right-sizing, utilization, the software levers below) therefore carries a facility multiplier, while chasing switch watts buys little in absolute terms - the fabric's real sustainability contribution is keeping accelerators busy (Latency and the straggler problem), because stranded GPU-hours dwarf the network's ~3% share. At the other end of the pipe, HRG (heat recovery generation) returns roughly 3% of facility load as generation - negligible at 100 MW (~3 MW) but material at multi-GW scale (30+ MW) - so heat recovery, like firm carbon-free supply (nuclear/SMR-adjacent siting), moves from gesture to a genuine siting-and-design requirement as the facility grows.
Green AI vs Red AI, and the five cost categories
Red AI chases accuracy at any compute cost (ever-bigger models, brute force); Green AI treats efficiency as a first-class goal - comparable results for less energy and spend. Framing AI spend across five cost categories - data, compute, development, utilization, and maintenance - shows where to optimise: compute and utilization usually dominate, so keeping accelerators busy and right-sized is both the green move and the cheap one (it ties back to TCO in Cost, ROI, and scalability of AI infrastructure).
Reducing AI power: hardware and software levers
Beyond utilisation, AI power is cut on two fronts. Hardware: prefer AI accelerators (TPU/NPU/ASIC) over CPUs for the same work; power capping limits peak draw; DVFS (Dynamic Voltage and Frequency Scaling) trims voltage/clock to the workload; CRAM (compute-in-memory, still research) promises large reductions by avoiding logic<->memory data movement. Software (model optimisation): quantization (e.g. 8-bit instead of 32-bit) shrinks compute/memory; pruning removes redundant parameters; knowledge distillation trains a small "student" to mimic a large "teacher" so inference runs on far less. And heat is not a footnote: cooling alone is ~30-40% of a data centre's energy, so the cooling choice is itself a major sustainability lever.
Power delivery - conversion losses, DC bus, and fault-managed power
The electrical chain is a loss ladder of its own: every stage between grid and silicon costs percentage points - power-factor loss (10%+), AC/DC conversion (20-40%), transformer and phase-balance loss (10-20%), idle-load and PSU overhead (20%+), in-chassis fans (~15%) - an illustrative ~40% cumulative conversion loss across a legacy AC distribution. The structural observation is that the endpoints are already DC: GPUs, batteries, solar, and fuel cells are natively DC, so a 380VDC-class building bus removes double conversions and lets on-site generation and a BESS (battery energy storage system) attach directly. Fault-managed power (FMP) is what makes higher-voltage DC distribution touch-safe: delivery is actively monitored and faults are isolated selectively and fast, which also gives the power plane network-like operations - real-time fault management, dynamic load management, automated energy redistribution (AI-assisted operations - AIOps) - and enables retrofit power backhaul without conduit-heavy electrical work. The placement decision mirrors every other placement question: top-of-rack FMP with centralized BESS (finest fault isolation and per-rack ride-through, most units to manage) -> row-level FMP + BESS (the middle ground) -> centralized FMP + BESS (fewest units, longest DC runs, widest per-fault blast radius); HVDC covers long on-campus transmission. The vendor headline of ~50% operational savings combining liquid cooling with FMP is a lab illustration, but the direction - fewer conversions, DC-native sources and storage, actively managed distribution - is the design content.
Reading PUE honestly - and its water twin
PUE (facility power / IT power) has an accounting trap: server-internal fans count as IT load, which flatters air cooling. Fans consume roughly ~15% of server power in an air-cooled GPU server vs <5% with direct liquid cooling, so a "1.2-PUE" air-cooled hall can really spend ~1.45 W of total power per watt delivered to silicon, against ~1.15 for a ~1.1-PUE liquid design - compare total watts per chip-watt, not the PUE headline. The best figures come from chiller-less facility-water loops feeding direct-to-chip cold plates through a liquid-to-liquid exchange. The water twin is WUE (liters per kWh): evaporative free cooling buys PUE with water - a dry-climate site can burn several times the typical ~0.5 L/kWh - and regulators may then force dry chiller operation, trading the water back for a worse PUE. In a scenario, treat power, heat, and water as one coupled budget.
Recommendation / justification
Plan power and cooling to the rack density (liquid cooling - typically direct-to-chip - for dense GPU racks), and maximize utilization because efficiency, sustainability, and cost are the same lever. Consider PUE, heat reuse, and siting. Justify by the reality that power/cooling is often the binding constraint on GPU scale.
What would change this (mid-scenario twist)
- Rack density exceeds air-cooling limits -> direct-to-chip liquid (or immersion at the extreme).
- Sustainability/regulatory targets -> efficiency (utilization, PUE, renewable siting) becomes a hard requirement.
Validation checks
- Does the cooling method match the rack density (liquid for dense GPU racks)?
- Is GPU utilization high (idle accelerators = wasted power and capital)?
IPv6 / dual-stack note
Environmental/power design is independent of the IP stack.
Related
- AI-enabling hardware - GPU, DPU, SmartNIC
- Scale-up vs scale-out
- Optics and cabling selection
- Latency and the straggler problem
- AI-assisted operations - AIOps - the power plane grows a control plane (FMP telemetry and actuation)
Spaced repetition
Why does AI force liquid cooling?
AI racks draw 40-100+ kW vs ~5-15 kW traditional - far beyond air-cooling capacity - so dense GPU racks need direct-to-chip (or immersion) liquid cooling.
Why is GPU utilization a sustainability issue?
Idle GPUs waste power and capital, so keeping GPUs busy (minimizing stragglers) is simultaneously an efficiency, sustainability, and cost win - "effective use of accelerators."
Power and cooling matter for design because they are often [...].
Power and cooling matter for design because they are often the binding constraint on how many GPUs you can deploy.
Green AI vs Red AI, and the five AI cost categories?
Red AI maximises accuracy at any compute cost; Green AI makes efficiency a first-class goal. The five cost categories are data, compute, development, utilization, and maintenance - compute/utilization usually dominate, so right-sizing and high utilisation are the biggest levers.
Name the software model-optimization levers that cut AI power.
Quantization (lower numeric precision, e.g. 8-bit vs 32-bit), pruning (remove redundant parameters), and knowledge distillation (small student mimics a large teacher). On hardware: AI accelerators, power capping, DVFS, and (research) compute-in-memory CRAM. Cooling alone is ~30-40% of DC energy.
Besides cooling 40-100 kW AI racks, what is the opposite supply-side power constraint?
Many enterprise/colocation racks are capped at only ~8-15 kW (and in colo that is usually fixed), while an AI server can draw 1-2 kW+, so only a few fit - forcing you to upgrade the feed (often infeasible) or spread servers across under-filled racks (stranded space and cost).
Why size rack power to peak rather than average, and around what?
GPUs spike hard under load, so plan for the server's maximum draw (tripping a breaker or starving cooling is worse than headroom); GPUs are the dominant power draw, so size the power/cooling budget around them.
Why can PUE flatter an air-cooled AI hall, and what should you compare instead?
Server fans count as IT load (~15% of server power air-cooled vs <5% liquid-cooled), so compare total watts per watt delivered to silicon - roughly 1.45 for a nominal 1.2-PUE air design vs ~1.15 for a ~1.1-PUE direct-liquid design; and check WUE, since evaporative free cooling buys PUE with water.
A brownfield hall hits GPU hot spots but cannot re-plumb the floor - what is the first cooling move?
A rear-door heat exchanger fed by a locationally deployed CDU (the ~20-40 kW/rack band) - it solves spot problems without layout changes and is the on-ramp toward direct-to-chip.
The cooling ladder runs air -> rear-door heat exchanger -> direct-to-chip -> single-phase immersion -> immersion+DtC, at roughly [...].
The cooling ladder runs air -> rear-door heat exchanger -> direct-to-chip -> single-phase immersion -> immersion+DtC, at roughly 8-16 -> 20-40 -> 150 -> 250 -> 350+ kW/rack.
Why move an AI facility toward a 380VDC bus with FMP?
GPUs, BESS, solar, and fuel cells are natively DC, so a DC bus removes lossy AC/DC double conversions (legacy chain ~40% cumulative, illustrative); fault-managed power makes higher-voltage DC touch-safe with fast selective fault isolation and enables retrofit power backhaul without heavy electrical work.
FMP/BESS placement (top-of-rack vs row-level vs centralized) trades [...].
FMP/BESS placement (top-of-rack vs row-level vs centralized) trades fault-isolation granularity and local ride-through against the number of managed units and DC run length.
What is the facility cascade for compute-side power savings, and what does it imply for the network?
Roughly 1.55 W saved at the facility per 1 W saved at the compute load (its cooling/distribution overhead scales away); the network is only ~3% of facility power, so the fabric's sustainability lever is accelerator utilization and job completion, not switch watts.
[...] returns ~3% of facility load as generation - negligible at 100 MW, material (30+ MW) at multi-GW scale.
HRG (heat recovery generation) returns ~3% of facility load as generation - negligible at 100 MW, material (30+ MW) at multi-GW scale.
What does the AI POD's 30 kW rack budget force, and what follows for the row plan?
At ~12.5 kW per 8-GPU node, two nodes fill a 42RU rack; the switch-plus-management rack runs ~20 kW/23RU leaving ~10 kW/17RU for storage - so plan racks and power to the eventual cluster (the >512-GPU re-cable inflection) and give each row its own FE leaf pair to keep cabling row-local.
Sources
- Cisco AI POD for Enterprise Training and Fine-Tuning Design Guide (Jan 2026) - worked 30 kW rack layout, two 12.5 kW nodes per rack, row-local FE leaves, >512-GPU inflection; full 94-page pass 2026-07-26 (user-supplied PDF).
- Cisco AI/ML blueprint; data-center liquid-cooling and PUE/efficiency references.
- Cisco Live BRKGRN-2419, Sustainability vs. Infrastructure Requirements (Liljenstolpe/Bean/Ebermann) - cooling ladder, FMP/DC distribution, efficiency cascade, HRG.