A30 Specifications
Cut-down A100 at 165 W with MIG. Half the A100 price for mainstream inference where 24 GB is enough.
Buy it for
Mainstream servers needing HBM2 bandwidth and MIG in a 165 W envelope — modest inference fleets where a full A100 is over-specified.
Look elsewhere if
You want capacity. 24 GB puts it well below the A100, and the L40S beats it on most inference work for a similar price.
Will it fit your system?
Passively cooled — needs server airflow
This card has no fans. It relies on a chassis forcing air through its heatsink. In a normal desktop it will throttle within minutes and can overheat outright. It needs a rack server with high-static-pressure fans, or an aftermarket shroud and blower.
- Form factor
- PCIe dual-slot
- Cooling
- passive (no fans)
- Power connectors
- 1 × 8-pin EPS (CPU-style — not a PCIe 8-pin)
- Board power
- 165 W
- Suggested PSU
- 550 W or better
- Interface
- PCIe Gen 4
PSU figure assumes one card plus a mainstream CPU and drives, with headroom for transient spikes — modern cards can draw well above their rated TDP for milliseconds, which is what trips an undersized supply.
Supported formats & features
What the A30 does and does not support in hardware.
Blackwell-generation feature — B200 has it
Predates hardware FP8; use INT8 or BF16 instead — H100 has it
Wider exponent than FP16 — fewer overflow issues when training
Drop-in FP32 speedup with no code changes
Quantised inference supported
INT4 tensor operations supported
Precision must be managed manually
200 GB/s GPU-to-GPU
Compute-only die — no ray tracing hardware
Corrects single-bit errors — expected for long training runs
What fits in memory
Largest model that fits on one A30, assuming ~20% of the 24 GB is reserved for KV cache and activations.
| Model | FP16 / BF16 | INT4 / FP4 |
|---|---|---|
| Llama 8B | Fits | Fits |
| Llama 70B | 8× GPUs | 2× GPUs |
| Mixtral 8x22B | 15× GPUs | 4× GPUs |
| Llama 405B | 43× GPUs | 11× GPUs |
Weights only. Long contexts, large batch sizes and optimiser state during training all need substantially more — full fine-tuning typically needs 3–4× the weight memory.
Workload character
Plenty of bandwidth per unit of compute. Memory-bound work — single-stream token decode above all — runs close to its ceiling, and compute is usually what runs out first.
Cost to run
Electricity for one card at 100% duty, including cooling and PSU overhead (2,024 kWh/year).
At the US average rate, three years of continuous operation costs $728 — roughly 15% of the card's $5,000 launch price, before any cooling or hosting fees.
Availability & market
Never sold in volume and awkward to place between the A10 and A100. Thin used supply; verify support before committing.
Memory
- Capacity
- 24 GB
- Type
- HBM2
- Bandwidth
- 933 GB/s
- Bus width
- 3,072-bit
- GB per TFLOPMemory relative to compute — higher favours large models
- 0.145
Compute
- FP64
- 5.2 TFLOPS
- FP32
- 10.3 TFLOPS
- TF32 dense
- 82 TFLOPS
- FP16 dense
- 165 TFLOPS
- FP16 sparseRequires structured sparsity; rarely reached in practice
- 330 TFLOPS
- FP8 dense
- —
- INT8 dense
- 330 TOPS
Silicon
- Codename
- GA100
- Process node
- TSMC 7N
- Transistors
- 54.2B
- Die size
- 826 mm²
- CUDA cores
- —
- Tensor cores
- —
- SM count
- 56
- Boost clock
- 1,440 MHz
Platform & power
- TDP
- 165 W
- TFLOPS per wattDense FP16 ÷ TDP — drives your power bill at scale
- 1.00
- Form factor
- PCIe dual-slot
- Cooling
- passive
- Interconnect
- NVLink
- Interconnect speed
- 200 GB/s
- PCIe generation
- Gen 4
- MIG
- Yes — up to 4 instances
- Launch
- 2021-04
- MSRP at launch
- $5,000
A30 — common questions
Answered from the specifications above. Questions this card's data cannot answer with confidence are left out rather than guessed.
- How much VRAM does the A30 have?
- The A30 has 24 GB of HBM2 memory with 933 GB/s of bandwidth. In practice plan on roughly 19 GB for weights: KV cache, activations and fragmentation take the rest, and long contexts or large batches take considerably more.
- What size model can the A30 run?
- On a single A30, about 9B parameters at FP16, or roughly 38B quantised to FP4. Anything larger has to be split across several GPUs. These are weight-only figures — a long context window can consume tens of gigabytes on its own.
- Can you put the A30 in a desktop PC?
- Only with added cooling. The A30 is a passively cooled card — it has no fans at all and depends on a rack chassis forcing air through its heatsink. Dropped into an ordinary desktop it will throttle within minutes and can overheat outright. It needs either a rack server with high-static-pressure fans or an aftermarket shroud and blower.
- How much power does the A30 use?
- The A30 is rated at 165 W board power. It is fed through 1 × 8-pin EPS (CPU-style — not a PCIe 8-pin). For a single-card system, a 550 W supply or better leaves headroom for the CPU, drives and the transient spikes that trip undersized units.
- How much does it cost to run the A30?
- At the US commercial average of $0.12/kWh, running one A30 continuously costs about $243 a year in electricity. That figure includes cooling and power-supply overhead at a factor of 1.4 over board power, and over three years comes to roughly 15% of what the card cost at launch.
- Does the A30 support MIG?
- Yes. The A30 can be partitioned into up to 4 Multi-Instance GPU slices, each with isolated memory and compute. This is what makes it practical to share one card across several users or tenants without them affecting each other's latency.
- Does the A30 support FP8?
- No. The A30 has no hardware FP8 path. FP8-quantised checkpoints will still load, but the values are upcast before they reach the tensor cores, so you get the memory saving without the speedup.
- Does the A30 support FP4?
- No. The A30 predates hardware FP4. You can run 4-bit quantised weights on it and the memory saving is real, but they are dequantised before compute, so there is no arithmetic speedup.
- Does the A30 support INT8?
- Yes. The A30 supports INT8 acceleration at 330 dense TOPS. INT8 remains the most widely supported quantised inference format across frameworks.
- Does the A30 support BF16?
- Yes. The A30 supports bfloat16. Its wider exponent range makes training far less prone to the overflow that plagues FP16, which is why most current training recipes assume it.
- Does the A30 have NVLink?
- Yes. The A30 supports NVLink (NVLink) at 200 GB/s, so GPU-to-GPU traffic bypasses PCIe. This matters for tensor-parallel serving, where every token crosses the link.
- Does the A30 have RT cores?
- No. The A30 is a compute-only part with no ray-tracing hardware. For rendering or virtual workstations, look at the L40S or an RTX workstation card instead.
- Does the A30 have ECC memory?
- Yes. The A30 has ECC, so single-bit memory errors are corrected rather than silently corrupting a long run.
- Is the A30 worth buying?
- Buy it for: Mainstream servers needing HBM2 bandwidth and MIG in a 165 W envelope — modest inference fleets where a full A100 is over-specified. Look elsewhere if: You want capacity. 24 GB puts it well below the A100, and the L40S beats it on most inference work for a similar price.