L4 Specifications
72 W single-slot with FP8. The efficiency pick for inference density where power and slots are the constraint.
Buy it for
Dense, power-constrained inference. 72 W single-slot means many cards per server without a power upgrade.
Look elsewhere if
You need raw throughput or more than 24 GB — this is an efficiency part, not a fast one.
Will it fit your system?
Passively cooled — needs server airflow
This card has no fans. It relies on a chassis forcing air through its heatsink. In a normal desktop it will throttle within minutes and can overheat outright. It needs a rack server with high-static-pressure fans, or an aftermarket shroud and blower.
- Form factor
- PCIe single-slot
- Cooling
- passive (no fans)
- Power connectors
- None — slot-powered
- Board power
- 72 W
- Suggested PSU
- 400 W or better
- Interface
- PCIe Gen 4
PSU figure assumes one card plus a mainstream CPU and drives, with headroom for transient spikes — modern cards can draw well above their rated TDP for milliseconds, which is what trips an undersized supply.
Supported formats & features
What the L4 does and does not support in hardware.
Blackwell-generation feature — B200 has it
E4M3 and E5M2 — the standard format for modern inference
Wider exponent than FP16 — fewer overflow issues when training
Drop-in FP32 speedup with no code changes
Quantised inference supported
Not yet confirmed against vendor documentation.
Precision must be managed manually
Multi-GPU traffic goes over PCIe — a real limit for distributed training
Hardware ray tracing for rendering workloads
Corrects single-bit errors — expected for long training runs
What fits in memory
Largest model that fits on one L4, assuming ~20% of the 24 GB is reserved for KV cache and activations.
| Model | FP16 / BF16 | FP8 |
|---|---|---|
| Llama 8B | Fits | Fits |
| Llama 70B | 8× GPUs | 4× GPUs |
| Mixtral 8x22B | 15× GPUs | 8× GPUs |
| Llama 405B | 43× GPUs | 22× GPUs |
Weights only. Long contexts, large batch sizes and optimiser state during training all need substantially more — full fine-tuning typically needs 3–4× the weight memory.
Workload character
Compute outruns bandwidth. Strong on compute-bound phases such as prefill and training, but batch-1 decode will leave FLOPS idle waiting on memory.
Cost to run
Electricity for one card at 100% duty, including cooling and PSU overhead (883 kWh/year).
At the US average rate, three years of continuous operation costs $318 — roughly 13% of the card's $2,500 launch price, before any cooling or hosting fees.
Where it sits in the line
Availability & market
Cheap, widely stocked and fits almost any server. The usual answer for scaling out small-model inference.
Memory
- Capacity
- 24 GB
- Type
- GDDR6
- Bandwidth
- 300 GB/s
- Bus width
- 192-bit
- GB per TFLOPMemory relative to compute — higher favours large models
- 0.198
Compute
- FP64
- —
- FP32
- 30.3 TFLOPS
- TF32 dense
- 60.5 TFLOPS
- FP16 dense
- 121 TFLOPS
- FP16 sparseRequires structured sparsity; rarely reached in practice
- 242 TFLOPS
- FP8 dense
- 242 TFLOPS
- INT8 dense
- 242 TOPS
Silicon
- Codename
- AD104
- Process node
- TSMC 4N
- Transistors
- 35.8B
- Die size
- 294 mm²
- CUDA cores
- 7,424
- Tensor cores
- 240
- SM count
- 60
- Boost clock
- 2,040 MHz
Platform & power
- TDP
- 72 W
- TFLOPS per wattDense FP16 ÷ TDP — drives your power bill at scale
- 1.68
- Form factor
- PCIe single-slot
- Cooling
- passive
- Interconnect
- None
- Interconnect speed
- —
- PCIe generation
- Gen 4
- MIG
- No
- Launch
- 2023-03
- MSRP at launch
- $2,500
L4 — common questions
Answered from the specifications above. Questions this card's data cannot answer with confidence are left out rather than guessed.
- How much VRAM does the L4 have?
- The L4 has 24 GB of GDDR6 memory with 300 GB/s of bandwidth. In practice plan on roughly 19 GB for weights: KV cache, activations and fragmentation take the rest, and long contexts or large batches take considerably more.
- What size model can the L4 run?
- On a single L4, about 9B parameters at FP16, or roughly 19B quantised to FP8. Anything larger has to be split across several GPUs, which is slow on this card because it has no NVLink. These are weight-only figures — a long context window can consume tens of gigabytes on its own.
- Can you put the L4 in a desktop PC?
- Only with added cooling. The L4 is a passively cooled card — it has no fans at all and depends on a rack chassis forcing air through its heatsink. Dropped into an ordinary desktop it will throttle within minutes and can overheat outright. It needs either a rack server with high-static-pressure fans or an aftermarket shroud and blower.
- How much power does the L4 use?
- The L4 is rated at 72 W board power. It is fed through None — slot-powered. For a single-card system, a 400 W supply or better leaves headroom for the CPU, drives and the transient spikes that trip undersized units.
- How much does it cost to run the L4?
- At the US commercial average of $0.12/kWh, running one L4 continuously costs about $106 a year in electricity. That figure includes cooling and power-supply overhead at a factor of 1.4 over board power, and over three years comes to roughly 13% of what the card cost at launch.
- Does the L4 support FP8?
- Yes. The L4 has hardware FP8 tensor support, so FP8 weights and activations run at full tensor-core rate — 242 dense TFLOPS. This is the format most current inference servers default to for large models.
- Does the L4 support FP4?
- No. The L4 predates hardware FP4. You can run 4-bit quantised weights on it and the memory saving is real, but they are dequantised before compute, so there is no arithmetic speedup.
- Does the L4 support INT8?
- Yes. The L4 supports INT8 acceleration at 242 dense TOPS. INT8 remains the most widely supported quantised inference format across frameworks.
- Does the L4 support BF16?
- Yes. The L4 supports bfloat16. Its wider exponent range makes training far less prone to the overflow that plagues FP16, which is why most current training recipes assume it.
- Does the L4 have NVLink?
- No. The L4 has no NVLink. Multi-GPU work goes over PCIe Gen 4, which is an order of magnitude slower and makes tensor parallelism across cards expensive.
- Does the L4 have RT cores?
- Yes. The L4 includes dedicated ray-tracing cores, so it is usable for rendering and visualisation as well as compute.
- Does the L4 have ECC memory?
- Yes. The L4 has ECC, so single-bit memory errors are corrected rather than silently corrupting a long run.
- Is the L4 worth buying?
- Buy it for: Dense, power-constrained inference. 72 W single-slot means many cards per server without a power upgrade. Look elsewhere if: You need raw throughput or more than 24 GB — this is an efficiency part, not a fast one.