H100 PCIe Specifications
Cut-down H100 for standard servers. Half the TDP of SXM but ~24% less FP16 and 40% less bandwidth.
Buy it for
Dropping Hopper into standard PCIe servers without an HGX baseboard. Good single-GPU inference.
Look elsewhere if
You plan to scale past two GPUs — the bridge tops out at 600 GB/s and bandwidth is 40% below SXM.
Will it fit your system?
Passively cooled — needs server airflow
This card has no fans. It relies on a chassis forcing air through its heatsink. In a normal desktop it will throttle within minutes and can overheat outright. It needs a rack server with high-static-pressure fans, or an aftermarket shroud and blower.
- Form factor
- PCIe dual-slot
- Cooling
- passive (no fans)
- Power connectors
- 1x 16-pin CEM5 (12VHPWR)
- Board power
- 350 W
- Suggested PSU
- 850 W or better
- Interface
- PCIe Gen 5
PSU figure assumes one card plus a mainstream CPU and drives, with headroom for transient spikes — modern cards can draw well above their rated TDP for milliseconds, which is what trips an undersized supply.
Supported formats & features
What the H100 PCIe does and does not support in hardware.
Blackwell-generation feature — B200 has it
E4M3 and E5M2 — the standard format for modern inference
Wider exponent than FP16 — fewer overflow issues when training
Drop-in FP32 speedup with no code changes
Quantised inference supported
Not yet confirmed against vendor documentation.
Per-layer automatic FP8 precision management
600 GB/s GPU-to-GPU
Compute-only die — no ray tracing hardware
Corrects single-bit errors — expected for long training runs
What fits in memory
Largest model that fits on one H100 PCIe, assuming ~20% of the 80 GB is reserved for KV cache and activations.
| Model | FP16 / BF16 | FP8 |
|---|---|---|
| Llama 8B | Fits | Fits |
| Llama 70B | 3× GPUs | 2× GPUs |
| Mixtral 8x22B | 5× GPUs | 3× GPUs |
| Llama 405B | 13× GPUs | 7× GPUs |
Weights only. Long contexts, large batch sizes and optimiser state during training all need substantially more — full fine-tuning typically needs 3–4× the weight memory.
Workload character
Compute outruns bandwidth. Strong on compute-bound phases such as prefill and training, but batch-1 decode will leave FLOPS idle waiting on memory.
Cost per unit of compute
Breakeven counts hardware cost against rental only. Power, cooling, hosting and depreciation push the real crossover substantially later — see colocation pricing for what hosting actually costs.
Cost to run
Electricity for one card at 100% duty, including cooling and PSU overhead (4,292 kWh/year).
At the US average rate, three years of continuous operation costs $1,545 — roughly 6% of the card's $25,000 launch price, before any cooling or hosting fees.
Where it sits in the line
A newer part existing does not make this one a bad buy — it makes it cheaper. What matters is price per unit of the throughput you actually need.
Availability & market
Easier to source than SXM and fits normal 2U servers. Commands a premium over A100 on the used market.
Memory
- Capacity
- 80 GB
- Type
- HBM2e
- Bandwidth
- 2,000 GB/s
- Bus width
- 5,120-bit
- GB per TFLOPMemory relative to compute — higher favours large models
- 0.106
Compute
- FP64
- 26 TFLOPS
- FP32
- 51 TFLOPS
- TF32 dense
- 378 TFLOPS
- FP16 dense
- 756 TFLOPS
- FP16 sparseRequires structured sparsity; rarely reached in practice
- 1,513 TFLOPS
- FP8 dense
- 1,513 TFLOPS
- INT8 dense
- 1,513 TOPS
Silicon
- Codename
- GH100
- Process node
- TSMC 4N
- Transistors
- 80B
- Die size
- 814 mm²
- CUDA cores
- 14,592
- Tensor cores
- 456
- SM count
- 114
- Boost clock
- 1,755 MHz
Platform & power
- TDP
- 350 W
- TFLOPS per wattDense FP16 ÷ TDP — drives your power bill at scale
- 2.16
- Form factor
- PCIe dual-slot
- Cooling
- passive
- Interconnect
- NVLink bridge
- Interconnect speed
- 600 GB/s
- PCIe generation
- Gen 5
- MIG
- Yes — up to 7 instances
- Launch
- 2022-09
- MSRP at launch
- $25,000
H100 PCIe — common questions
Answered from the specifications above. Questions this card's data cannot answer with confidence are left out rather than guessed.
- How much VRAM does the H100 PCIe have?
- The H100 PCIe has 80 GB of HBM2e memory with 2,000 GB/s of bandwidth. In practice plan on roughly 64 GB for weights: KV cache, activations and fragmentation take the rest, and long contexts or large batches take considerably more.
- What size model can the H100 PCIe run?
- On a single H100 PCIe, about 32B parameters at FP16, or roughly 64B quantised to FP8. Anything larger has to be split across several GPUs. These are weight-only figures — a long context window can consume tens of gigabytes on its own.
- Can you put the H100 PCIe in a desktop PC?
- Only with added cooling. The H100 PCIe is a passively cooled card — it has no fans at all and depends on a rack chassis forcing air through its heatsink. Dropped into an ordinary desktop it will throttle within minutes and can overheat outright. It needs either a rack server with high-static-pressure fans or an aftermarket shroud and blower.
- How much power does the H100 PCIe use?
- The H100 PCIe is rated at 350 W board power. It is fed through 1x 16-pin CEM5 (12VHPWR). For a single-card system, a 850 W supply or better leaves headroom for the CPU, drives and the transient spikes that trip undersized units.
- How much does it cost to run the H100 PCIe?
- At the US commercial average of $0.12/kWh, running one H100 PCIe continuously costs about $515 a year in electricity. That figure includes cooling and power-supply overhead at a factor of 1.4 over board power, and over three years comes to roughly 6% of what the card cost at launch.
- Does the H100 PCIe support MIG?
- Yes. The H100 PCIe can be partitioned into up to 7 Multi-Instance GPU slices, each with isolated memory and compute. This is what makes it practical to share one card across several users or tenants without them affecting each other's latency.
- Does the H100 PCIe support FP8?
- Yes. The H100 PCIe has hardware FP8 tensor support, so FP8 weights and activations run at full tensor-core rate — 1,513 dense TFLOPS. This is the format most current inference servers default to for large models.
- Does the H100 PCIe support FP4?
- No. The H100 PCIe predates hardware FP4. You can run 4-bit quantised weights on it and the memory saving is real, but they are dequantised before compute, so there is no arithmetic speedup.
- Does the H100 PCIe support INT8?
- Yes. The H100 PCIe supports INT8 acceleration at 1,513 dense TOPS. INT8 remains the most widely supported quantised inference format across frameworks.
- Does the H100 PCIe support BF16?
- Yes. The H100 PCIe supports bfloat16. Its wider exponent range makes training far less prone to the overflow that plagues FP16, which is why most current training recipes assume it.
- Does the H100 PCIe have NVLink?
- Yes. The H100 PCIe supports NVLink (NVLink bridge) at 600 GB/s, so GPU-to-GPU traffic bypasses PCIe. This matters for tensor-parallel serving, where every token crosses the link.
- Does the H100 PCIe have RT cores?
- No. The H100 PCIe is a compute-only part with no ray-tracing hardware. For rendering or virtual workstations, look at the L40S or an RTX workstation card instead.
- Does the H100 PCIe have ECC memory?
- Yes. The H100 PCIe has ECC, so single-bit memory errors are corrected rather than silently corrupting a long run.
- What replaced the H100 PCIe?
- The H100 NVL (2023) is its successor, with 94 GB of HBM3 at 3,900 GB/s. It is about 1.2x faster in dense FP16. On the used market that gap is what sets a fair price for the H100 PCIe.
- Is the H100 PCIe worth buying?
- Buy it for: Dropping Hopper into standard PCIe servers without an HGX baseboard. Good single-GPU inference. Look elsewhere if: You plan to scale past two GPUs — the bridge tops out at 600 GB/s and bandwidth is 40% below SXM.