MI300X Specifications
AMD flagship. 192 GB HBM3 fits models that need two H100s. The gap is software: ROCm has closed a lot but CUDA remains the default path.
Buy it for
Large-model inference where 192 GB on one card removes the need for tensor parallelism.
Look elsewhere if
Your stack is CUDA-bound. ROCm has closed much of the gap but porting is still real work.
Will it fit your system?
This is not a card you can install
OAM is a socketed module that mounts to an OEM baseboard, not a PCIe slot. You buy it inside a complete system — it cannot go into an existing desktop or a standard server under any circumstances.
- Form factor
- OAM
- Cooling
- passive
- Power connectors
- OAM socket — requires an OAM baseboard
- Board power
- 750 W
- Interface
- PCIe Gen 5
Supported formats & features
What the MI300X does and does not support in hardware.
Blackwell-generation feature — B200 has it
E4M3 and E5M2 — the standard format for modern inference
Wider exponent than FP16 — fewer overflow issues when training
Not available on this architecture
Quantised inference supported
INT4 tensor operations supported
Precision must be managed manually
Multi-GPU traffic goes over PCIe — a real limit for distributed training
Compute-only die — no ray tracing hardware
Corrects single-bit errors — expected for long training runs
What fits in memory
Largest model that fits on one MI300X, assuming ~20% of the 192 GB is reserved for KV cache and activations.
| Model | FP16 / BF16 | FP8 | INT4 / FP4 |
|---|---|---|---|
| Llama 8B | Fits | Fits | Fits |
| Llama 70B | Fits | Fits | Fits |
| Mixtral 8x22B | 2× GPUs | Fits | Fits |
| Llama 405B | 6× GPUs | 3× GPUs | 2× GPUs |
Weights only. Long contexts, large batch sizes and optimiser state during training all need substantially more — full fine-tuning typically needs 3–4× the weight memory.
Workload character
Bandwidth and compute are matched closely enough that neither dominates across mixed training and inference.
Cost to run
Electricity for one card at 100% duty, including cooling and PSU overhead (9,198 kWh/year).
At the US average rate, three years of continuous operation costs $3,311 — roughly 18% of the card's $18,000 launch price, before any cooling or hosting fees.
Where it sits in the line
A newer part existing does not make this one a bad buy — it makes it cheaper. What matters is price per unit of the throughput you actually need.
Availability & market
Sold through OEM systems. Often materially cheaper per GB than NVIDIA if your software runs on it.
Memory
- Capacity
- 192 GB
- Type
- HBM3
- Bandwidth
- 5,300 GB/s
- Bus width
- 8,192-bit
- GB per TFLOPMemory relative to compute — higher favours large models
- 0.147
Compute
- FP64
- 163.4 TFLOPS
- FP32
- 163.4 TFLOPS
- TF32 dense
- 653.7 TFLOPS
- FP16 dense
- 1,307.4 TFLOPS
- FP16 sparseRequires structured sparsity; rarely reached in practice
- 2,614.9 TFLOPS
- FP8 dense
- 2,614.9 TFLOPS
- INT8 dense
- 2,614.9 TOPS
Silicon
- Codename
- Aqua Vanjaram
- Process node
- TSMC 5nm + 6nm chiplet
- Transistors
- 153B
- Die size
- 1,017 mm²
- CUDA cores
- —
- Tensor cores
- 1,216
- SM count
- 304
- Boost clock
- 2,100 MHz
Platform & power
- TDP
- 750 W
- TFLOPS per wattDense FP16 ÷ TDP — drives your power bill at scale
- 1.74
- Form factor
- OAM
- Cooling
- passive
- Interconnect
- Infinity Fabric
- Interconnect speed
- 896 GB/s
- PCIe generation
- Gen 5
- MIG
- Yes — up to 8 instances
- Launch
- 2023-12
- MSRP at launch
- $18,000
MI300X — common questions
Answered from the specifications above. Questions this card's data cannot answer with confidence are left out rather than guessed.
- How much VRAM does the MI300X have?
- The MI300X has 192 GB of HBM3 memory with 5,300 GB/s of bandwidth. In practice plan on roughly 153 GB for weights: KV cache, activations and fragmentation take the rest, and long contexts or large batches take considerably more.
- What size model can the MI300X run?
- On a single MI300X, about 76B parameters at FP16, or roughly 307B quantised to FP4. Anything larger has to be split across several GPUs, which is slow on this card because it has no NVLink. These are weight-only figures — a long context window can consume tens of gigabytes on its own.
- Can you put the MI300X in a desktop PC?
- No. The MI300X is a OAM module, not a PCIe card. It bolts to an OEM baseboard and is only sold inside a complete system, so there is no way to install one in a desktop or a standard server.
- How much power does the MI300X use?
- The MI300X is rated at 750 W board power. It draws that from the baseboard rather than from PSU cables, so there is nothing to wire up yourself.
- How much does it cost to run the MI300X?
- At the US commercial average of $0.12/kWh, running one MI300X continuously costs about $1,104 a year in electricity. That figure includes cooling and power-supply overhead at a factor of 1.4 over board power, and over three years comes to roughly 18% of what the card cost at launch.
- Does the MI300X support MIG?
- Yes. The MI300X can be partitioned into up to 8 Multi-Instance GPU slices, each with isolated memory and compute. This is what makes it practical to share one card across several users or tenants without them affecting each other's latency.
- Does the MI300X support FP8?
- Yes. The MI300X has hardware FP8 tensor support, so FP8 weights and activations run at full tensor-core rate — 2,614.9 dense TFLOPS. This is the format most current inference servers default to for large models.
- Does the MI300X support FP4?
- No. The MI300X predates hardware FP4. You can run 4-bit quantised weights on it and the memory saving is real, but they are dequantised before compute, so there is no arithmetic speedup.
- Does the MI300X support INT8?
- Yes. The MI300X supports INT8 acceleration at 2,614.9 dense TOPS. INT8 remains the most widely supported quantised inference format across frameworks.
- Does the MI300X support BF16?
- Yes. The MI300X supports bfloat16. Its wider exponent range makes training far less prone to the overflow that plagues FP16, which is why most current training recipes assume it.
- Does the MI300X have NVLink?
- No. The MI300X has no NVLink. Multi-GPU work goes over PCIe Gen 5, which is an order of magnitude slower and makes tensor parallelism across cards expensive.
- Does the MI300X have RT cores?
- No. The MI300X is a compute-only part with no ray-tracing hardware. For rendering or virtual workstations, look at the L40S or an RTX workstation card instead.
- Does the MI300X have ECC memory?
- Yes. The MI300X has ECC, so single-bit memory errors are corrected rather than silently corrupting a long run.
- What replaced the MI300X?
- The MI325X (2024) is its successor, with 256 GB of HBM3e at 6,000 GB/s. At the same dense FP16 throughput, what changed is memory rather than compute. On the used market that gap is what sets a fair price for the MI300X.
- Is the MI300X worth buying?
- Buy it for: Large-model inference where 192 GB on one card removes the need for tensor parallelism. Look elsewhere if: Your stack is CUDA-bound. ROCm has closed much of the gap but porting is still real work.