MI325X Specifications
MI300X with 256 GB HBM3e at 6 TB/s. Highest memory capacity of any single accelerator shipping.
Buy it for
Serving very large models on few GPUs. 256 GB of HBM3e per module is the largest capacity available, so models that need several H100s fit on fewer MI325X.
Look elsewhere if
Your stack is not already working on ROCm. The gap has narrowed for inference, but anything depending on custom CUDA kernels or niche libraries will need porting work.
Will it fit your system?
This is not a card you can install
OAM is a socketed module that mounts to an OEM baseboard, not a PCIe slot. You buy it inside a complete system — it cannot go into an existing desktop or a standard server under any circumstances.
- Form factor
- OAM
- Cooling
- passive
- Power connectors
- None — OAM module, powered by the baseboard
- Board power
- 1000 W
- Interface
- PCIe Gen 5
Supported formats & features
What the MI325X does and does not support in hardware.
Blackwell-generation feature — B200 has it
E4M3 and E5M2 — the standard format for modern inference
Wider exponent than FP16 — fewer overflow issues when training
Not available on this architecture
Quantised inference supported
INT4 tensor operations supported
Precision must be managed manually
Multi-GPU traffic goes over PCIe — a real limit for distributed training
Compute-only die — no ray tracing hardware
Corrects single-bit errors — expected for long training runs
What fits in memory
Largest model that fits on one MI325X, assuming ~20% of the 256 GB is reserved for KV cache and activations.
| Model | FP16 / BF16 | FP8 | INT4 / FP4 |
|---|---|---|---|
| Llama 8B | Fits | Fits | Fits |
| Llama 70B | Fits | Fits | Fits |
| Mixtral 8x22B | 2× GPUs | Fits | Fits |
| Llama 405B | 4× GPUs | 2× GPUs | Fits |
Weights only. Long contexts, large batch sizes and optimiser state during training all need substantially more — full fine-tuning typically needs 3–4× the weight memory.
Workload character
Bandwidth and compute are matched closely enough that neither dominates across mixed training and inference.
Cost to run
Electricity for one card at 100% duty, including cooling and PSU overhead (12,264 kWh/year).
At the US average rate, three years of continuous operation costs $4,415 — roughly 20% of the card's $22,000 launch price, before any cooling or hosting fees.
Where it sits in the line
Availability & market
Sold as complete OEM platforms, not as cards. Available through fewer clouds than NVIDIA, so check your provider offers it before designing around it.
Memory
- Capacity
- 256 GB
- Type
- HBM3e
- Bandwidth
- 6,000 GB/s
- Bus width
- 8,192-bit
- GB per TFLOPMemory relative to compute — higher favours large models
- 0.196
Compute
- FP64
- 163.4 TFLOPS
- FP32
- 163.4 TFLOPS
- TF32 dense
- 653.7 TFLOPS
- FP16 dense
- 1,307.4 TFLOPS
- FP16 sparseRequires structured sparsity; rarely reached in practice
- 2,614.9 TFLOPS
- FP8 dense
- 2,614.9 TFLOPS
- INT8 dense
- 2,614.9 TOPS
Silicon
- Codename
- Aqua Vanjaram
- Process node
- TSMC 5nm + 6nm chiplet
- Transistors
- 153B
- Die size
- 1,017 mm²
- CUDA cores
- —
- Tensor cores
- 1,216
- SM count
- 304
- Boost clock
- 2,100 MHz
Platform & power
- TDP
- 1,000 W
- TFLOPS per wattDense FP16 ÷ TDP — drives your power bill at scale
- 1.31
- Form factor
- OAM
- Cooling
- passive
- Interconnect
- Infinity Fabric
- Interconnect speed
- 896 GB/s
- PCIe generation
- Gen 5
- MIG
- Yes — up to 8 instances
- Launch
- 2024-Q4
- MSRP at launch
- $22,000
MI325X — common questions
Answered from the specifications above. Questions this card's data cannot answer with confidence are left out rather than guessed.
- How much VRAM does the MI325X have?
- The MI325X has 256 GB of HBM3e memory with 6,000 GB/s of bandwidth. In practice plan on roughly 204 GB for weights: KV cache, activations and fragmentation take the rest, and long contexts or large batches take considerably more.
- What size model can the MI325X run?
- On a single MI325X, about 102B parameters at FP16, or roughly 409B quantised to FP4. Anything larger has to be split across several GPUs, which is slow on this card because it has no NVLink. These are weight-only figures — a long context window can consume tens of gigabytes on its own.
- Can you put the MI325X in a desktop PC?
- No. The MI325X is a OAM module, not a PCIe card. It bolts to an OEM baseboard and is only sold inside a complete system, so there is no way to install one in a desktop or a standard server.
- How much power does the MI325X use?
- The MI325X is rated at 1000 W board power. It draws that from the baseboard rather than from PSU cables, so there is nothing to wire up yourself.
- How much does it cost to run the MI325X?
- At the US commercial average of $0.12/kWh, running one MI325X continuously costs about $1,472 a year in electricity. That figure includes cooling and power-supply overhead at a factor of 1.4 over board power, and over three years comes to roughly 20% of what the card cost at launch.
- Does the MI325X support MIG?
- Yes. The MI325X can be partitioned into up to 8 Multi-Instance GPU slices, each with isolated memory and compute. This is what makes it practical to share one card across several users or tenants without them affecting each other's latency.
- Does the MI325X support FP8?
- Yes. The MI325X has hardware FP8 tensor support, so FP8 weights and activations run at full tensor-core rate — 2,614.9 dense TFLOPS. This is the format most current inference servers default to for large models.
- Does the MI325X support FP4?
- No. The MI325X predates hardware FP4. You can run 4-bit quantised weights on it and the memory saving is real, but they are dequantised before compute, so there is no arithmetic speedup.
- Does the MI325X support INT8?
- Yes. The MI325X supports INT8 acceleration at 2,614.9 dense TOPS. INT8 remains the most widely supported quantised inference format across frameworks.
- Does the MI325X support BF16?
- Yes. The MI325X supports bfloat16. Its wider exponent range makes training far less prone to the overflow that plagues FP16, which is why most current training recipes assume it.
- Does the MI325X have NVLink?
- No. The MI325X has no NVLink. Multi-GPU work goes over PCIe Gen 5, which is an order of magnitude slower and makes tensor parallelism across cards expensive.
- Does the MI325X have RT cores?
- No. The MI325X is a compute-only part with no ray-tracing hardware. For rendering or virtual workstations, look at the L40S or an RTX workstation card instead.
- Does the MI325X have ECC memory?
- Yes. The MI325X has ECC, so single-bit memory errors are corrected rather than silently corrupting a long run.
- Is the MI325X worth buying?
- Buy it for: Serving very large models on few GPUs. 256 GB of HBM3e per module is the largest capacity available, so models that need several H100s fit on fewer MI325X. Look elsewhere if: Your stack is not already working on ROCm. The gap has narrowed for inference, but anything depending on custom CUDA kernels or niche libraries will need porting work.