A100 SXM 80GB Specifications
Ampere workhorse. No FP8, so inference throughput lags Hopper badly, but plentiful on the used market.
Buy it for
Budget training and fine-tuning. Plentiful on the secondary market now that Hopper has displaced it.
Look elsewhere if
Inference throughput matters — no FP8 means Hopper is roughly 3x faster per watt on modern serving stacks.
Will it fit your system?
This is not a card you can install
SXM4 is a socketed module that mounts to an OEM baseboard, not a PCIe slot. You buy it inside a complete system — it cannot go into an existing desktop or a standard server under any circumstances.
- Form factor
- SXM4
- Cooling
- passive
- Power connectors
- SXM4 socket — no cables; requires an OEM baseboard
- Board power
- 400 W
- Interface
- PCIe Gen 4
Supported formats & features
What the A100 SXM 80GB does and does not support in hardware.
Blackwell-generation feature — B200 has it
Predates hardware FP8; use INT8 or BF16 instead — H100 has it
Wider exponent than FP16 — fewer overflow issues when training
Drop-in FP32 speedup with no code changes
Quantised inference supported
INT4 tensor operations supported
Precision must be managed manually
600 GB/s GPU-to-GPU
Compute-only die — no ray tracing hardware
Corrects single-bit errors — expected for long training runs
What fits in memory
Largest model that fits on one A100 SXM 80GB, assuming ~20% of the 80 GB is reserved for KV cache and activations.
| Model | FP16 / BF16 | INT4 / FP4 |
|---|---|---|
| Llama 8B | Fits | Fits |
| Llama 70B | 3× GPUs | Fits |
| Mixtral 8x22B | 5× GPUs | 2× GPUs |
| Llama 405B | 13× GPUs | 4× GPUs |
Weights only. Long contexts, large batch sizes and optimiser state during training all need substantially more — full fine-tuning typically needs 3–4× the weight memory.
Workload character
Plenty of bandwidth per unit of compute. Memory-bound work — single-stream token decode above all — runs close to its ceiling, and compute is usually what runs out first.
Cost per unit of compute
Breakeven counts hardware cost against rental only. Power, cooling, hosting and depreciation push the real crossover substantially later — see colocation pricing for what hosting actually costs.
Cost to run
Electricity for one card at 100% duty, including cooling and PSU overhead (4,906 kWh/year).
At the US average rate, three years of continuous operation costs $1,766 — roughly 12% of the card's $15,000 launch price, before any cooling or hosting fees.
Where it sits in the line
A newer part existing does not make this one a bad buy — it makes it cheaper. What matters is price per unit of the throughput you actually need.
Availability & market
The value pick on the used market. Ex-datacenter units are common; check warranty status and hours before buying.
Memory
- Capacity
- 80 GB
- Type
- HBM2e
- Bandwidth
- 2,039 GB/s
- Bus width
- 5,120-bit
- GB per TFLOPMemory relative to compute — higher favours large models
- 0.256
Compute
- FP64
- 9.7 TFLOPS
- FP32
- 19.5 TFLOPS
- TF32 dense
- 156 TFLOPS
- FP16 dense
- 312 TFLOPS
- FP16 sparseRequires structured sparsity; rarely reached in practice
- 624 TFLOPS
- FP8 dense
- —
- INT8 dense
- 624 TOPS
Silicon
- Codename
- GA100
- Process node
- TSMC 7N
- Transistors
- 54.2B
- Die size
- 826 mm²
- CUDA cores
- 6,912
- Tensor cores
- 432
- SM count
- 108
- Boost clock
- 1,410 MHz
Platform & power
- TDP
- 400 W
- TFLOPS per wattDense FP16 ÷ TDP — drives your power bill at scale
- 0.78
- Form factor
- SXM4
- Cooling
- passive
- Interconnect
- NVLink 3.0
- Interconnect speed
- 600 GB/s
- PCIe generation
- Gen 4
- MIG
- Yes — up to 7 instances
- Launch
- 2020-11
- MSRP at launch
- $15,000
A100 SXM 80GB — common questions
Answered from the specifications above. Questions this card's data cannot answer with confidence are left out rather than guessed.
- How much VRAM does the A100 SXM 80GB have?
- The A100 SXM 80GB has 80 GB of HBM2e memory with 2,039 GB/s of bandwidth. In practice plan on roughly 64 GB for weights: KV cache, activations and fragmentation take the rest, and long contexts or large batches take considerably more.
- What size model can the A100 SXM 80GB run?
- On a single A100 SXM 80GB, about 32B parameters at FP16, or roughly 128B quantised to FP4. Anything larger has to be split across several GPUs. These are weight-only figures — a long context window can consume tens of gigabytes on its own.
- Can you put the A100 SXM 80GB in a desktop PC?
- No. The A100 SXM 80GB is a SXM4 module, not a PCIe card. It bolts to an OEM baseboard and is only sold inside a complete system, so there is no way to install one in a desktop or a standard server.
- How much power does the A100 SXM 80GB use?
- The A100 SXM 80GB is rated at 400 W board power. It draws that from the baseboard rather than from PSU cables, so there is nothing to wire up yourself.
- How much does it cost to run the A100 SXM 80GB?
- At the US commercial average of $0.12/kWh, running one A100 SXM 80GB continuously costs about $589 a year in electricity. That figure includes cooling and power-supply overhead at a factor of 1.4 over board power, and over three years comes to roughly 12% of what the card cost at launch.
- Does the A100 SXM 80GB support MIG?
- Yes. The A100 SXM 80GB can be partitioned into up to 7 Multi-Instance GPU slices, each with isolated memory and compute. This is what makes it practical to share one card across several users or tenants without them affecting each other's latency.
- Does the A100 SXM 80GB support FP8?
- No. The A100 SXM 80GB has no hardware FP8 path. FP8-quantised checkpoints will still load, but the values are upcast before they reach the tensor cores, so you get the memory saving without the speedup.
- Does the A100 SXM 80GB support FP4?
- No. The A100 SXM 80GB predates hardware FP4. You can run 4-bit quantised weights on it and the memory saving is real, but they are dequantised before compute, so there is no arithmetic speedup.
- Does the A100 SXM 80GB support INT8?
- Yes. The A100 SXM 80GB supports INT8 acceleration at 624 dense TOPS. INT8 remains the most widely supported quantised inference format across frameworks.
- Does the A100 SXM 80GB support BF16?
- Yes. The A100 SXM 80GB supports bfloat16. Its wider exponent range makes training far less prone to the overflow that plagues FP16, which is why most current training recipes assume it.
- Does the A100 SXM 80GB have NVLink?
- Yes. The A100 SXM 80GB supports NVLink (NVLink 3.0) at 600 GB/s, so GPU-to-GPU traffic bypasses PCIe. This matters for tensor-parallel serving, where every token crosses the link.
- Does the A100 SXM 80GB have RT cores?
- No. The A100 SXM 80GB is a compute-only part with no ray-tracing hardware. For rendering or virtual workstations, look at the L40S or an RTX workstation card instead.
- Does the A100 SXM 80GB have ECC memory?
- Yes. The A100 SXM 80GB has ECC, so single-bit memory errors are corrected rather than silently corrupting a long run.
- What replaced the A100 SXM 80GB?
- The H100 SXM (2022) is its successor, with 80 GB of HBM3 at 3,350 GB/s. It is about 3.2x faster in dense FP16. On the used market that gap is what sets a fair price for the A100 SXM 80GB.
- Is the A100 SXM 80GB worth buying?
- Buy it for: Budget training and fine-tuning. Plentiful on the secondary market now that Hopper has displaced it. Look elsewhere if: Inference throughput matters — no FP8 means Hopper is roughly 3x faster per watt on modern serving stacks.