Liquid cooling loop commissioning field guide for AI data centers

Liquid cooling loop commissioning, and why it is not air commissioning

Liquid cooling loop commissioning is the process of proving a direct-to-chip cooling system is clean, leak-tight, balanced, and controlled before coolant and live hardware go into it. The loop carries heat off the chip cold plates with water or a water-glycol mix instead of air, because AI and GPU racks now pull 40, 80, and past 100 kW, more heat than you can move with any volume of air you can physically push through a rack. Liquid holds far more heat per unit volume, so it carries the same kilowatts in a fraction of the flow and gets the cooling within inches of the silicon.

The reason this commissioning is its own discipline, separate from the air-side test-and-balance work, is what a mistake costs. A bad air balance leaves a hall inefficient. A bad liquid loop leaks coolant onto an electronics cabinet full of GPUs that can run six and seven figures per rack, or it sends a slug of construction debris into a cold plate and chokes the cooling on the one chip that needed it most. The whole job is protecting expensive hardware from leaks and contamination.

So the work reorders around two questions the air side never had to ask. Is the loop clean enough to touch a cold plate, and will a leak be caught before it ever reaches the hardware. Everything in this guide serves those two. The cooling pillar covers the air-side architecture and the chilled water plant that still sits behind every liquid loop, so this guide stays on the loop itself.

The architecture: cold plate, manifold, CDU, and two loops

A direct-to-chip system is two loops joined by a heat exchanger, and getting the names straight is the first step to commissioning it. The technology cooling system, the TCS, is the secondary loop. It is the clean, conditioned loop that runs from the coolant distribution unit out to the racks, up the rack manifold, and through the cold plates clamped to the CPUs and GPUs. The facility water system, the FWS, is the primary loop. It is the building chilled water that the chiller plant makes and the cooling tower rejects.

The two loops do not mix. The coolant distribution unit, the CDU, sits between them with a heat exchanger that passes heat from the TCS to the FWS without passing the water. That isolation is the point. The facility water can be dirtier, warmer, and at building pressure, while the TCS stays clean, conditioned, and at the controlled pressure and temperature the cold plates need. A liquid-to-liquid CDU rejects to facility water this way. A liquid-to-air CDU rejects the TCS heat straight to room air through a coil instead, which suits a smaller deployment or a hall with no facility water loop.

Inside the rack, the manifold is the vertical distribution header that splits the supply coolant to each node and collects the return. Quick-disconnects join the node to the manifold so a server can be pulled without draining the loop. The cold plate itself is a sealed block with internal micro-channels pressed against the chip, and those channels are the narrowest, most fragile part of the entire system. Everything upstream exists to deliver clean coolant to them at the right flow and temperature.

Direct-to-chip vs rear-door vs immersion: where does each fit?

Direct-to-chip, rear-door heat exchangers, and immersion are three liquid architectures, and they layer in by density and by how much of the rack the liquid touches. Direct-to-chip, also called cold plate, runs coolant through a plate on the CPU and GPU and takes the high-density heat straight off the silicon while air still handles the rest of the rack, the memory, drives, and power supplies. It is the dominant choice for current AI and HPC racks because it targets the hottest parts and bolts onto an otherwise conventional server.

A rear-door heat exchanger hangs a coil on the back of the rack and pulls heat out of the exhaust air before it enters the room. It is the lightest touch, a way to push an air-cooled hall to higher density without re-plumbing the servers, but it tops out lower than cold plate because it is still cooling air. Immersion drops the whole server into a bath of dielectric fluid, single-phase or two-phase, and is the densest option with no server fans at all, at the cost of a completely different serviceability and fluid-handling regime. The immersion cooling work is its own acceptance scope, cross-linked by topic rather than covered here.

Most large AI deployments today are direct-to-chip, often hybrid, with cold plates on the GPUs and contained air or a rear door handling the balance of the rack heat. That hybrid is why the air-side commissioning does not go away when liquid arrives. You are commissioning a liquid loop and an air path at the same rack, and a cold plate that runs perfectly does nothing for the memory module the air was supposed to cool.

The coolant: what runs in the secondary loop

The coolant is not just water, and it is not yours to choose on the jobsite. The most common secondary-loop fluid for single-phase direct-to-chip is a propylene-glycol-and-water mix, commonly around 25 percent propylene glycol by volume, often written PG25, with a corrosion-inhibitor package matched to the metals in the loop. ASHRAE TC 9.9 and the Open Compute Project have largely converged on a PG25-class fluid as the reference for single-phase cold-plate loops, but the equipment manufacturer's specified coolant governs, every time, over any rule of thumb.

The glycol earns its place for two reasons beyond freeze protection. At roughly 25 percent it drops the freeze point to about minus 10 C, which protects coolant sitting in a cold dry-cooler outdoors, and it suppresses biological growth far better than plain water in the warm, dark, nutrient-poor environment of a cold-plate loop. The inhibitor package is the part people underrate. It is tuned to the specific copper, brass, stainless, and elastomers in the system, and it depletes over time, which is why the coolant becomes a maintenance item the owner inherits, not a fill-and-forget.

Treated or deionized water shows up in some designs, and two-phase dielectric fluids appear in immersion and a few cold-plate systems where any leak must be non-conductive. The chemistry matters because the coolant lives in contact with micro-channels and dissimilar metals for years. Wrong fluid, wrong concentration, or a missing inhibitor package and you get corrosion, scale, or biofilm that narrows the channels and starves the chip. Confirm the coolant, the concentration, and the inhibitor against the manufacturer's specification and the OCP or ASHRAE guidance the design referenced, and do not let a contractor top off a glycol loop with plain water.

Why flush the liquid cooling loop before connecting cold plates?

Flushing the loop to a cleanliness target before any cold plate is connected is the single most important step in liquid cooling commissioning, and it is the one most likely to be rushed. New piping, manifolds, and hoses come full of construction debris: weld slag, pipe scale, cutting oil, thread sealant, plastic shavings, and the fine grit that gets into anything built on a jobsite. Send that into a cold plate and it lodges in micro-channels that can be a fraction of a millimeter wide, restricting flow to the one chip that can least afford it. A choked cold plate does not announce itself until the chip throttles or cooks.

The flush is staged. You start with a coarse filter to mobilize and capture the heavy construction debris, circulating at a velocity high enough to scour the pipe walls, commonly in the range of 3 to 5 ft/s, then step down to progressively finer filtration until the loop holds a measured cleanliness target. Targets vary by manufacturer and by how fine the cold-plate channels are. Some quick-disconnect and component specs call for no particles above 100 microns, while micro-channel cold plates often demand filtration to 50 microns or finer, and many direct-to-chip projects drive the final filtration into the low single-digit micron range. The Open Compute Project guidance points at roughly a 5 micron target as a practical compromise, but the cold-plate manufacturer's number is the one that governs.

Two rules keep crews honest here. Flush with the procedure fluid the spec calls for, often a conditioned water that meets a documented conductivity and microbiological limit, not whatever is in the yard hose, and verify the cleanliness by sampling against the acceptance criteria, not by eye. Then, and only then, you connect the cold plates. The discipline is the same idea as flushing a chilled water loop before it touches a coil, which the cooling pillar treats as a hold point, except here the consequence lands on hardware that cannot be flushed back out.

How do you pressure-test a liquid cooling loop?

You pressure-test the loop to find every leak before coolant goes in and long before hardware does. The proof test pressurizes the assembled piping, manifolds, and CDU above the system's working pressure, holds it, and watches for any pressure decay that betrays a leak. A common approach is a hydrostatic test with water or the procedure fluid, held at a multiple of the design working pressure for a documented duration while the gauge is watched and every joint is checked. The test pressure, the medium, and the hold time come from the project specification and the applicable piping code, not from habit.

Some teams use a pneumatic or a combined pressure-decay test on portions of the system, particularly where introducing water before the cleanliness step is undesirable, but air under pressure stores far more energy than water and a pneumatic test on the wrong component is a safety event, so it is done deliberately and per a procedure, never casually. Where the piping falls under a pressure-piping code, the relevant ASME B31 section by topic and the project spec set the test parameters and acceptance. Confirm which code section applies to the system you have before you set a number.

Hold the test long enough for temperature to stabilize, because a falling gauge can be a real leak or just the test water cooling and contracting, and the two look identical for the first few minutes. Trend the pressure and the fluid temperature together so you can tell a leak from a thermal artifact. The chilled-water hydrostatic test is the same physics one loop upstream, and the HydroTest workflow walks through holding pressure, logging the decay, and distinguishing a true leak from a thermal settle.

How do you leak-test a liquid cooling loop?

You prove leak detection works before you trust the loop near hardware, because in liquid cooling the leak strategy is the whole game over the IT. The detection layers stack. Rope, or linear, leak-sensing cable runs along the piping and under the manifolds and reports both that a leak happened and roughly where along the run. Point, or spot, sensors sit at the low points and in the drip trays under the CDU and the rack, where coolant collects first. Drip trays and containment under every wet connection give a leak somewhere to go that is not the electronics below.

The commissioning is not just confirming the sensors are installed. You test them. Wet a rope sensor with a measured drop and confirm it alarms and reports the zone. Trip a point sensor in a tray and confirm the alarm reaches the BMS and, where the design calls for it, that the CDU initiates the programmed response, an alarm, a pump action, or an isolation. A leak detection system that nobody verified by actually wetting a sensor is a system you are hoping works, and hope is not a commissioning record.

The reflex on the floor is to fixate on finding leaks. The better instinct is to assume there will be a leak someday and prove that when it happens, it is detected, contained, and contained before it reaches a powered cabinet. That is why drip-tray slope, sensor placement at the true low points, and the tested alarm path matter as much as the pressure test that tries to prevent the leak in the first place.

Negative-pressure (sub-ambient) loops

A negative-pressure loop turns the leak problem inside out. Instead of running the coolant above atmospheric pressure, where a breach pushes coolant out onto the hardware, the system runs the loop below ambient so any breach draws air in rather than coolant out. The pump pulls the fluid through under vacuum, and if a hose, a fitting, or a cold plate develops a crack, the pressure difference injects air bubbles into the loop instead of spraying coolant across a GPU tray.

That changes how you detect and how you commission. The CDU watches for the signature of a leak as air ingestion or as a degradation of the vacuum it is holding, not as coolant on a tray, and it can pull the ingested air back to a reservoir and vent it. Commissioning a sub-ambient design means verifying the loop holds its vacuum, that the air-separation and venting work, and that an induced leak shows up as the air-ingress alarm the design promises. It is a different acceptance test from a positive-pressure loop, and the procedure has to match the architecture in front of you.

Negative-pressure is not universal, and it carries its own tradeoffs around achievable flow and pump selection, so plenty of deployments run conventional positive-pressure loops with layered detection instead. The point for the commissioning agent is to know which one you are testing, because the failure mode you are proving against, coolant out versus air in, is the opposite in each.

Flow and pressure balancing across the racks

Balancing sets the coolant flow each rack and each node actually gets, because a cold plate starved of flow throttles the chip just as surely as a leak would. The number to hit is the flow rate per node the cold-plate manufacturer specifies, often given in liters per minute or gallons per minute at a stated pressure drop across the plate. Too little flow and the chip runs hot. Too much and you are spending pump energy and pressure budget you do not have, or starving the rack next door.

The lever is the pressure drop, the dP, across each path. The CDU pump develops a head, and that head is consumed by the manifold, the quick-disconnects, the hoses, and the dP across the cold plate itself. A long rack or an unbalanced manifold can leave the top nodes short while the bottom nodes run rich, the same way a tall riser unbalances a hydronic system. You balance with the flow-control and balancing devices the design provides, then verify the per-node flow and the dP against the spec, not against the average across the rack.

The other number that lives here is the approach temperature at the CDU heat exchanger, the gap between the facility water entering and the TCS coolant leaving. A widening approach means the heat exchanger is fouling or losing capacity, and it shows up as a coolant supply temperature creeping up even though the facility water is fine. Record the design approach at commissioning so operations has a baseline to trend against, because the approach is one of the first things to drift as a loop ages.

What coolant temperature should reach the chip?

The coolant supplied to the cold plate is warmer than most people expect, and that is by design. ASHRAE TC 9.9 classifies liquid-cooling facility supply temperatures into water classes, recently renamed to put the temperature in the name: W17, W27, W32, W40, W45, and W+ correspond to maximum supply fluid temperatures of about 17, 27, 32, 40, 45, and above 45 C, with a common lower limit around 2 C across the classes. The class the design targets sets how warm the water can be and still cool the chips.

Warm-water cooling is the whole reason liquid pays off on energy. A chip will cool fine on coolant in the 30s of degrees C because the temperature difference between a 60 to 90 C junction and a 30-something C coolant is still large, and heat flows on the difference. Running the loop warm instead of cold means the facility can reject heat to the outside air for far more hours of the year with an economizer or a dry cooler, and in many climates skip mechanical refrigeration for the liquid load entirely. Colder coolant buys nothing for the chip and costs chiller energy.

So the commissioning target is the supply temperature the design and the silicon vendor agreed on, held stable, not the coldest the plant can make. Confirm the water class the project designed to and the maximum coolant supply temperature the chip vendor allows, because the chip's junction-temperature limit and the cold-plate thermal resistance, not the ASHRAE class alone, set the real ceiling. The class numbers and the edition move, so verify them against the current ASHRAE TC 9.9 thermal guidelines and the equipment documents before citing them on a submittal.

Quick-disconnects and servicing a live rack

Quick-disconnects, the QDs, are what let a technician pull a node from a running rack without draining the loop or shutting the row down. A dripless QD seals both halves as they part, so disconnecting a server spills a drop at most instead of a stream, and reconnecting it does not pull air into the loop. On a hall that has to stay up, the QD is what makes liquid cooling serviceable at all.

The QD is also a leak path and a contamination path, which is why it gets attention at commissioning. Every coupling is a sealing surface and an elastomer that has to match the coolant chemistry, and every connect-disconnect cycle is a chance to ingest air or introduce a particle. The cleanliness spec exists partly because debris in the loop scores QD seals and makes them weep. Confirm the QDs are the specified part, rated for the working pressure and the coolant, and that the connections were made clean and seated fully.

The field reality is that QDs are where small leaks start on an aging loop, not in the middle of a brazed pipe run. A weeping QD under a manifold drips onto whatever is below it, which is exactly why the drip trays and point sensors live under the wet connections. When you commission the leak detection, the QD locations are the spots you most want covered, because they are the spots most likely to need it later.

CDU commissioning: redundancy, controls, and the BMS

The CDU is the heart of the secondary loop, and commissioning it is more than confirming it pumps. It holds the pumps, the heat exchanger, the filtration, the controls, and usually the leak-detection logic for the loop it serves. The redundancy is commonly N+1 on the pumps, meaning one more pump than the load needs, so a pump can fail or be serviced and the racks keep their flow. Prove it by failing a pump and confirming the standby picks up and holds the per-rack flow, not just by reading the nameplate.

The controls are where a CDU is made or lost. It modulates pump speed and the facility-water valve to hold the coolant supply temperature and the loop pressure or flow the design calls for, and it has to ride out a facility-water upset or a load swing without letting the coolant supply temperature wander out of band. Test the control loop the way it will actually be challenged: step the load, drop the facility water, and watch whether the CDU holds supply temperature and pressure within the design limits.

The BMS integration is the part that gets skipped and bites later. Every alarm the CDU can raise, leak detected, pump fault, filter differential pressure high, supply temperature high, flow low, has to actually reach the BMS and read correctly, and the critical ones have to drive the response the design specifies. Point-to-point verify the alarms end to end, because an alarm that lives only on the CDU's local screen is an alarm nobody sees at 3 a.m. The electrical side of that integrated control, the power that has to be there for the CDU to run through an event, is the power QA pillar's scope, and on a real facility the two get tested together.

Materials, corrosion, and the loop the owner inherits

A liquid loop is a slow chemistry experiment running for years, and the commissioning baseline is what tells the owner whether it is aging normally. The metals and elastomers in the loop have to be compatible with the coolant and with each other. ASHRAE TC 9.9 guidance and OCP cold-plate requirements converge on a short list of compatible materials, commonly copper, brass, 316L stainless, and EPDM elastomers, but the manufacturer's compatibility list governs the specific system. Mix in an incompatible metal, a zinc-plated fitting or an aluminum component the inhibitor was not formulated for, and galvanic corrosion starts dropping particles into the loop.

Biological growth is the other slow killer. A warm, low-flow, nutrient-poor loop is still a place biofilm will grow if the chemistry lets it, and biofilm narrows micro-channels and fouls the heat exchanger the same way scale does. The glycol concentration and the inhibitor package are part of what holds it back, which is why coolant chemistry is a recurring maintenance item, not a one-time fill. Sample the coolant at commissioning to set the baseline for pH, conductivity, inhibitor level, and biological count, against the manufacturer's acceptance criteria.

What the owner inherits is real and worth naming at turnover. The coolant degrades, the inhibitor depletes, filters load up and need changing, and the heat exchanger approach drifts as it fouls. A loop handed over without a chemistry baseline, a filter-change interval, and a coolant-sampling schedule is a loop that will quietly degrade until a chip throttles and nobody knows why. The maintenance plan is part of the commissioning deliverable, not an afterthought for operations to invent later.

What does the integrated test prove for a liquid loop?

The integrated test puts the liquid loop under load, real or simulated, and runs it through the failure scenarios to prove it carries the heat when something goes wrong, not just on a calm commissioning afternoon. You load the racks with IT load or with load banks or heaters that mimic the chip heat, bring the loop to its design supply temperature and flow, and then start breaking things on purpose. The point is to find the gap between the design intent and the as-built loop while there is still time and nobody's production workload is riding on it.

The scenarios are the ones that actually happen. Fail a CDU pump and confirm the N+1 standby holds flow with the racks at load. Drop the facility water and watch how long the loop rides through before the coolant supply temperature climbs out of band, because a liquid loop has thermal mass but not much, and a chip at 100 kW heats fast when the heat stops leaving. Trip a leak sensor under load and confirm the alarm and the programmed response fire. Lose utility power and confirm the cooling rides through the same gap the power plant has to ride through.

That last one is why the liquid integrated test does not stand alone. Cooling cannot ride through a power event the power side has not proven, so on a real facility the mechanical and electrical integrated tests run together. The power QA pillar covers the electrical half, the utility-loss transfer and the generator and UPS that have to carry the CDU through the gap, and the cooling pillar covers the chilled water plant behind the CDU. The liquid loop is the last link in that chain, and it is the one sitting closest to the hardware.

Field example: balancing a CDU at the as-found numbers

A liquid-to-liquid CDU feeding a row of direct-to-chip GPU racks came up at commissioning with the bottom nodes cool and the top nodes throttling under load. The CDU was making its design coolant supply temperature, around 32 C, and the facility water was on spec, so the first instinct was that the CDU was undersized. The per-node flow numbers said otherwise.

Measured flow at the top nodes was running well under the cold-plate manufacturer's specified rate while the bottom nodes ran above it, a classic vertical imbalance across the rack manifold. The dP across the top cold plates was low because the flow was short, and those chips were riding their throttle while the room average looked fine. The heat exchanger approach was on target, so the heat exchanger was not the problem. The distribution was.

Balancing the manifold with the flow-control devices brought every node within the specified flow band, the top-of-rack throttling stopped, and the same CDU carried the row at full load inside the supply-temperature spec. No capacity was added. The flow had been there all along, pooled at the bottom of the rack where it was not needed, the same way air pools in the wrong aisle when nobody balances the floor. The commissioning record then captured the as-balanced per-node flow as the baseline operations would trend against.

What to document

A liquid loop that was proven but never documented hands operations a system with no baseline, and the first warm chip becomes a guessing game. The record is what tells the next engineer whether a reading is a new problem or how the loop has always run. Capture the loop identity, the coolant and its verified chemistry, the flush and cleanliness result, the pressure test parameters and outcome, the per-rack and per-node flows and pressure drops, the coolant supply temperature and the heat exchanger approach, the leak-detection test results, and the signoff.

Two records carry the most weight later. The cleanliness verification proves the loop was clean before the cold plates went on, which is the question you cannot re-answer once a plate is fouled, and the leak-detection test proves the system that protects the hardware was actually exercised. A turnover package missing either of those leaves the owner trusting that the most consequential steps got done.

Common mistakes

• Connecting cold plates to a loop that was never flushed to the cleanliness target, then choking a micro-channel with construction debris.
• Skipping or rushing the flush, or flushing with a fluid that does not meet the spec's conductivity and microbiological limits.
• Filling with the wrong coolant, the wrong glycol concentration, or topping a glycol loop off with plain water and stripping the inhibitor.
• Pressurizing or running the loop without a proof pressure test, so the first leak shows up over live hardware.
• Installing leak detection but never wetting a sensor to prove the alarm and the programmed response actually fire.
• Not knowing whether the loop is positive or negative pressure, and testing for the wrong failure mode.
• Balancing to the rack average instead of the per-node flow the cold-plate spec requires, leaving the top nodes starved.
• Calling an N+1 CDU redundant without failing a pump under load to prove the standby holds flow.
• Handing over the loop with no chemistry baseline, filter interval, or coolant-sampling schedule for operations to run against.

Field checklist

Standards and references

The central reference for the thermal side is ASHRAE Technical Committee 9.9, whose thermal guidelines and liquid-cooling guidance set the facility water classes, the W17 through W+ supply-temperature bands, and the water-quality and material-compatibility guidance the secondary loop is built around. The Open Compute Project publishes the cold-plate and liquid-cooling requirements many AI deployments design to, including guidance on propylene-glycol-based heat transfer fluids for single-phase cold-plate racks, cold-plate cleanliness and filtration targets, and pre-commission preparation of the technology cooling system. Name the OCP document by topic and confirm the current revision, since the documents revise on their own cycle.

The coolant and the cleanliness target are governed first by the equipment and cold-plate manufacturer's specification, which overrides any general guideline where it is stricter. The pressure test of the piping falls under the applicable pressure-piping code, commonly an ASME B31 section by topic, together with the project specification that sets the test pressure, medium, and hold. The commissioning process framework draws on the ASHRAE commissioning guidance, commonly Guideline 0, and where a facility is chasing a tier, the Uptime Institute standards drive the witnessed redundancy and integrated-test demonstrations.

Edition numbers, the exact water-class values, and the OCP revision levels move between cycles, so confirm them against the published documents and the IT and cold-plate manufacturers before citing them on a submittal. ASHRAE TC 9.9 and OCP give the framework; the manufacturer's coolant and cleanliness specifications and the project documents control the limits you commission to.

Units, terms, and acronyms

Liquid cooling commissioning carries vocabulary from HVAC piping, from the IT side, and from the chip vendors, and the same idea reads differently across a CDU submittal, a cold-plate datasheet, and an ASHRAE guideline. The terms below travel across the whole liquid scope.

CDU

Coolant distribution unit, the pumps, controls, and heat exchanger that isolate the secondary loop from facility water

TCS

Technology cooling system, the clean conditioned secondary loop from the CDU to the cold plates

FWS

Facility water system, the primary chilled water loop the CDU rejects heat into

GPM / LPM

Gallons or liters per minute, the coolant flow rate balanced per node against the cold-plate spec

dP

Pressure drop across a cold plate, manifold, or path, the lever you balance flow with

QD

Quick-disconnect, a dripless coupling that joins a node to the manifold for live service

W-class

ASHRAE TC 9.9 facility water temperature class (W17 to W+), naming the maximum supply fluid temperature in C

PG25

A roughly 25 percent propylene-glycol-and-water coolant with an inhibitor package, a common secondary-loop fluid

Approach temperature

The gap between facility water in and coolant out at the heat exchanger; a widening approach signals fouling

Cold plate

The micro-channel block clamped to the chip that takes heat into the coolant; its channels are the most fragile part of the loop

Related tools

Calculators and readiness checks for this work

Compare your options

FAQ

People also ask

• What is bypass airflow in a data center?
• How long does data center electrical commissioning take?
• What is a CDU in a data center?
• What is direct-to-chip liquid cooling?
• Why do data centers need liquid-cooling leak detection?