A40 Specifications
Passive 48 GB card for rack servers. Effectively an A6000 built for datacenter airflow.
Buy it for
Professional visualisation and rendering with a large frame buffer — 48 GB with ECC, display outputs, and vGPU certification for virtual workstations.
Look elsewhere if
Your workload is pure training throughput. GDDR6 gives it well under half the A100 bandwidth, and LLM work is usually bandwidth-bound.
Will it fit your system?
Passively cooled — needs server airflow
This card has no fans. It relies on a chassis forcing air through its heatsink. In a normal desktop it will throttle within minutes and can overheat outright. It needs a rack server with high-static-pressure fans, or an aftermarket shroud and blower.
- Form factor
- PCIe dual-slot
- Cooling
- passive (no fans)
- Power connectors
- 1 × 8-pin EPS (CPU-style — not a PCIe 8-pin)
- Board power
- 300 W
- Suggested PSU
- 750 W or better
- Interface
- PCIe Gen 4
PSU figure assumes one card plus a mainstream CPU and drives, with headroom for transient spikes — modern cards can draw well above their rated TDP for milliseconds, which is what trips an undersized supply.
Supported formats & features
What the A40 does and does not support in hardware.
Blackwell-generation feature — B200 has it
Predates hardware FP8; use INT8 or BF16 instead — H100 has it
Wider exponent than FP16 — fewer overflow issues when training
Drop-in FP32 speedup with no code changes
Quantised inference supported
INT4 tensor operations supported
Precision must be managed manually
112 GB/s GPU-to-GPU
Hardware ray tracing for rendering workloads
Corrects single-bit errors — expected for long training runs
What fits in memory
Largest model that fits on one A40, assuming ~20% of the 48 GB is reserved for KV cache and activations.
| Model | FP16 / BF16 | INT4 / FP4 |
|---|---|---|
| Llama 8B | Fits | Fits |
| Llama 70B | 4× GPUs | Fits |
| Mixtral 8x22B | 8× GPUs | 2× GPUs |
| Llama 405B | 22× GPUs | 6× GPUs |
Weights only. Long contexts, large batch sizes and optimiser state during training all need substantially more — full fine-tuning typically needs 3–4× the weight memory.
Workload character
Bandwidth and compute are matched closely enough that neither dominates across mixed training and inference.
Cost to run
Electricity for one card at 100% duty, including cooling and PSU overhead (3,679 kWh/year).
At the US average rate, three years of continuous operation costs $1,325 — roughly 29% of the card's $4,500 launch price, before any cooling or hosting fees.
Availability & market
Widely available used from VDI fleets. Largely superseded by the L40S, which is faster for the same slot and power.
Memory
- Capacity
- 48 GB
- Type
- GDDR6 ECC
- Bandwidth
- 696 GB/s
- Bus width
- 384-bit
- GB per TFLOPMemory relative to compute — higher favours large models
- 0.321
Compute
- FP64
- —
- FP32
- 37.4 TFLOPS
- TF32 dense
- 74.8 TFLOPS
- FP16 dense
- 149.7 TFLOPS
- FP16 sparseRequires structured sparsity; rarely reached in practice
- 299.4 TFLOPS
- FP8 dense
- —
- INT8 dense
- 299.4 TOPS
Silicon
- Codename
- GA102
- Process node
- Samsung 8N
- Transistors
- 28.3B
- Die size
- 628 mm²
- CUDA cores
- 10,752
- Tensor cores
- 336
- SM count
- 84
- Boost clock
- 1,740 MHz
Platform & power
- TDP
- 300 W
- TFLOPS per wattDense FP16 ÷ TDP — drives your power bill at scale
- 0.50
- Form factor
- PCIe dual-slot
- Cooling
- passive
- Interconnect
- NVLink
- Interconnect speed
- 112 GB/s
- PCIe generation
- Gen 4
- MIG
- No
- Launch
- 2020-10
- MSRP at launch
- $4,500
A40 — common questions
Answered from the specifications above. Questions this card's data cannot answer with confidence are left out rather than guessed.
- How much VRAM does the A40 have?
- The A40 has 48 GB of GDDR6 ECC memory with 696 GB/s of bandwidth. In practice plan on roughly 38 GB for weights: KV cache, activations and fragmentation take the rest, and long contexts or large batches take considerably more.
- What size model can the A40 run?
- On a single A40, about 19B parameters at FP16, or roughly 76B quantised to FP4. Anything larger has to be split across several GPUs. These are weight-only figures — a long context window can consume tens of gigabytes on its own.
- Can you put the A40 in a desktop PC?
- Only with added cooling. The A40 is a passively cooled card — it has no fans at all and depends on a rack chassis forcing air through its heatsink. Dropped into an ordinary desktop it will throttle within minutes and can overheat outright. It needs either a rack server with high-static-pressure fans or an aftermarket shroud and blower.
- How much power does the A40 use?
- The A40 is rated at 300 W board power. It is fed through 1 × 8-pin EPS (CPU-style — not a PCIe 8-pin). For a single-card system, a 750 W supply or better leaves headroom for the CPU, drives and the transient spikes that trip undersized units.
- How much does it cost to run the A40?
- At the US commercial average of $0.12/kWh, running one A40 continuously costs about $442 a year in electricity. That figure includes cooling and power-supply overhead at a factor of 1.4 over board power, and over three years comes to roughly 29% of what the card cost at launch.
- Does the A40 support FP8?
- No. The A40 has no hardware FP8 path. FP8-quantised checkpoints will still load, but the values are upcast before they reach the tensor cores, so you get the memory saving without the speedup.
- Does the A40 support FP4?
- No. The A40 predates hardware FP4. You can run 4-bit quantised weights on it and the memory saving is real, but they are dequantised before compute, so there is no arithmetic speedup.
- Does the A40 support INT8?
- Yes. The A40 supports INT8 acceleration at 299.4 dense TOPS. INT8 remains the most widely supported quantised inference format across frameworks.
- Does the A40 support BF16?
- Yes. The A40 supports bfloat16. Its wider exponent range makes training far less prone to the overflow that plagues FP16, which is why most current training recipes assume it.
- Does the A40 have NVLink?
- Yes. The A40 supports NVLink (NVLink) at 112 GB/s, so GPU-to-GPU traffic bypasses PCIe. This matters for tensor-parallel serving, where every token crosses the link.
- Does the A40 have RT cores?
- Yes. The A40 includes dedicated ray-tracing cores, so it is usable for rendering and visualisation as well as compute.
- Does the A40 have ECC memory?
- Yes. The A40 has ECC, so single-bit memory errors are corrected rather than silently corrupting a long run.
- Is the A40 worth buying?
- Buy it for: Professional visualisation and rendering with a large frame buffer — 48 GB with ECC, display outputs, and vGPU certification for virtual workstations. Look elsewhere if: Your workload is pure training throughput. GDDR6 gives it well under half the A100 bandwidth, and LLM work is usually bandwidth-bound.