GH200 Grace Hopper Specifications
Hopper die fused to a Grace ARM CPU over a 900 GB/s coherent link. The CPU memory becomes addressable by the GPU, which is the point — models larger than HBM stop thrashing over PCIe.
Buy it for
Workloads whose working set exceeds GPU memory. Grace and Hopper share a coherent 900 GB/s NVLink-C2C link, so the CPU's LPDDR5X acts as fast second-tier memory — recommender models and large-graph work benefit most.
Look elsewhere if
You want a drop-in accelerator. It is a CPU+GPU superchip that replaces the host, so it is a platform decision, not a card purchase.
Will it fit your system?
This is not a card you can install
Superchip module is a socketed module that mounts to an OEM baseboard, not a PCIe slot. You buy it inside a complete system — it cannot go into an existing desktop or a standard server under any circumstances.
- Form factor
- Superchip module
- Cooling
- liquid
- Power connectors
- None — Superchip module, powered by the system board
- Board power
- 1000 W
- Interface
- PCIe Gen 5
Supported formats & features
What the GH200 Grace Hopper does and does not support in hardware.
Blackwell-generation feature — B200 has it
E4M3 and E5M2 — the standard format for modern inference
Wider exponent than FP16 — fewer overflow issues when training
Drop-in FP32 speedup with no code changes
Quantised inference supported
Not yet confirmed against vendor documentation.
Per-layer automatic FP8 precision management
900 GB/s GPU-to-GPU
Compute-only die — no ray tracing hardware
Corrects single-bit errors — expected for long training runs
What fits in memory
Largest model that fits on one GH200 Grace Hopper, assuming ~20% of the 144 GB is reserved for KV cache and activations.
| Model | FP16 / BF16 | FP8 |
|---|---|---|
| Llama 8B | Fits | Fits |
| Llama 70B | 2× GPUs | Fits |
| Mixtral 8x22B | 3× GPUs | 2× GPUs |
| Llama 405B | 8× GPUs | 4× GPUs |
Weights only. Long contexts, large batch sizes and optimiser state during training all need substantially more — full fine-tuning typically needs 3–4× the weight memory.
Workload character
Bandwidth and compute are matched closely enough that neither dominates across mixed training and inference.
Cost to run
Electricity for one card at 100% duty, including cooling and PSU overhead (12,264 kWh/year).
Where it sits in the line
Availability & market
Sold only as complete systems from OEMs. Rentable from a handful of clouds; try before committing to the architecture.
Memory
- Capacity
- 144 GB
- Type
- HBM3e
- Bandwidth
- 4,900 GB/s
- Bus width
- 6,144-bit
- GB per TFLOPMemory relative to compute — higher favours large models
- 0.146
Compute
- FP64
- 34 TFLOPS
- FP32
- 67 TFLOPS
- TF32 dense
- 494.5 TFLOPS
- FP16 dense
- 989.5 TFLOPS
- FP16 sparseRequires structured sparsity; rarely reached in practice
- 1,979 TFLOPS
- FP8 dense
- 1,979 TFLOPS
- INT8 dense
- 1,979 TOPS
Silicon
- Codename
- GH200
- Process node
- TSMC 4N
- Transistors
- 80B
- Die size
- 814 mm²
- CUDA cores
- 16,896
- Tensor cores
- 528
- SM count
- 132
- Boost clock
- —
Platform & power
- TDP
- 1,000 W
- TFLOPS per wattDense FP16 ÷ TDP — drives your power bill at scale
- 0.99
- Form factor
- Superchip module
- Cooling
- liquid
- Interconnect
- NVLink-C2C
- Interconnect speed
- 900 GB/s
- PCIe generation
- Gen 5
- MIG
- Yes — up to 7 instances
- Launch
- 2024-Q1
- MSRP at launch
- —
GH200 Grace Hopper — common questions
Answered from the specifications above. Questions this card's data cannot answer with confidence are left out rather than guessed.
- How much VRAM does the GH200 Grace Hopper have?
- The GH200 Grace Hopper has 144 GB of HBM3e memory with 4,900 GB/s of bandwidth. In practice plan on roughly 115 GB for weights: KV cache, activations and fragmentation take the rest, and long contexts or large batches take considerably more.
- What size model can the GH200 Grace Hopper run?
- On a single GH200 Grace Hopper, about 57B parameters at FP16, or roughly 115B quantised to FP8. Anything larger has to be split across several GPUs. These are weight-only figures — a long context window can consume tens of gigabytes on its own.
- Can you put the GH200 Grace Hopper in a desktop PC?
- No. The GH200 Grace Hopper is a Superchip module module, not a PCIe card. It bolts to an OEM baseboard and is only sold inside a complete system, so there is no way to install one in a desktop or a standard server.
- How much power does the GH200 Grace Hopper use?
- The GH200 Grace Hopper is rated at 1000 W board power. It draws that from the baseboard rather than from PSU cables, so there is nothing to wire up yourself.
- How much does it cost to run the GH200 Grace Hopper?
- At the US commercial average of $0.12/kWh, running one GH200 Grace Hopper continuously costs about $1,472 a year in electricity. That figure includes cooling and power-supply overhead at a factor of 1.4 over board power.
- Does the GH200 Grace Hopper support MIG?
- Yes. The GH200 Grace Hopper can be partitioned into up to 7 Multi-Instance GPU slices, each with isolated memory and compute. This is what makes it practical to share one card across several users or tenants without them affecting each other's latency.
- Does the GH200 Grace Hopper support FP8?
- Yes. The GH200 Grace Hopper has hardware FP8 tensor support, so FP8 weights and activations run at full tensor-core rate — 1,979 dense TFLOPS. This is the format most current inference servers default to for large models.
- Does the GH200 Grace Hopper support FP4?
- No. The GH200 Grace Hopper predates hardware FP4. You can run 4-bit quantised weights on it and the memory saving is real, but they are dequantised before compute, so there is no arithmetic speedup.
- Does the GH200 Grace Hopper support INT8?
- Yes. The GH200 Grace Hopper supports INT8 acceleration at 1,979 dense TOPS. INT8 remains the most widely supported quantised inference format across frameworks.
- Does the GH200 Grace Hopper support BF16?
- Yes. The GH200 Grace Hopper supports bfloat16. Its wider exponent range makes training far less prone to the overflow that plagues FP16, which is why most current training recipes assume it.
- Does the GH200 Grace Hopper have NVLink?
- Yes. The GH200 Grace Hopper supports NVLink (NVLink-C2C) at 900 GB/s, so GPU-to-GPU traffic bypasses PCIe. This matters for tensor-parallel serving, where every token crosses the link.
- Does the GH200 Grace Hopper have RT cores?
- No. The GH200 Grace Hopper is a compute-only part with no ray-tracing hardware. For rendering or virtual workstations, look at the L40S or an RTX workstation card instead.
- Does the GH200 Grace Hopper have ECC memory?
- Yes. The GH200 Grace Hopper has ECC, so single-bit memory errors are corrected rather than silently corrupting a long run.
- Is the GH200 Grace Hopper worth buying?
- Buy it for: Workloads whose working set exceeds GPU memory. Grace and Hopper share a coherent 900 GB/s NVLink-C2C link, so the CPU's LPDDR5X acts as fast second-tier memory — recommender models and large-graph work benefit most. Look elsewhere if: You want a drop-in accelerator. It is a CPU+GPU superchip that replaces the host, so it is a platform decision, not a card purchase.