Compare / GPU
COMPARE COMPUTE.
Choose up to three GPUs. Suitability depends on the workload — use the reference model to see how each one fits.
Compare / Up to 3 GPUs
3 selected
Precision
Workload
Model fit uses an estimated 158.0 GB at 8K context, batch 1. Blue marks the strongest value in a row — suitability depends on your workload, so there is no universal “best GPU”.
| Spec | NVIDIAA100 80GB | NVIDIARTX 4090 | NVIDIAH100 SXM |
|---|---|---|---|
| VRAM | 80 GB | 24 GB | 80 GB |
| Memory type | HBM2e | GDDR6X | HBM3 |
| Memory bandwidth | 2.04 TB/s | 1.01 TB/s | 3.35 TB/s |
| GPU class | Datacenter | Consumer | Datacenter |
| Architecture | Ampere | Ada Lovelace | Hopper |
| Workload type | InferenceFine-tuningTraining | InferenceFine-tuningTraining | InferenceFine-tuningTraining |
| Model fit · Llama 70B FP16 | 4 × A100 80GB162.0 GB headroom | 8 × RTX 409034.0 GB headroom | 4 × H100 SXM162.0 GB headroom |
| Power | 400 W | 450 W | 700 W |
| Interconnect | NVLink 3 · 600 GB/s | PCIe 4.0 | NVLink 4 · 900 GB/s |
| Typical use | Training and serving of 7B–70B models | Developer workstation inference and small-model fine-tuning | Large-scale training and high-throughput serving |
NVIDIA
A100 80GB- VRAM
- 80 GB
- Memory type
- HBM2e
- Memory bandwidth
- 2.04 TB/s
- GPU class
- Datacenter
- Architecture
- Ampere
- Workload type
- InferenceFine-tuningTraining
- Model fit · Llama 70B FP16
- 4 × A100 80GB162.0 GB headroom
- Power
- 400 W
- Interconnect
- NVLink 3 · 600 GB/s
- Typical use
- Training and serving of 7B–70B models
NVIDIA
RTX 4090- VRAM
- 24 GB
- Memory type
- GDDR6X
- Memory bandwidth
- 1.01 TB/s
- GPU class
- Consumer
- Architecture
- Ada Lovelace
- Workload type
- InferenceFine-tuningTraining
- Model fit · Llama 70B FP16
- 8 × RTX 409034.0 GB headroom
- Power
- 450 W
- Interconnect
- PCIe 4.0
- Typical use
- Developer workstation inference and small-model fine-tuning
NVIDIA
H100 SXM- VRAM
- 80 GB
- Memory type
- HBM3
- Memory bandwidth
- 3.35 TB/s
- GPU class
- Datacenter
- Architecture
- Hopper
- Workload type
- InferenceFine-tuningTraining
- Model fit · Llama 70B FP16
- 4 × H100 SXM162.0 GB headroom
- Power
- 700 W
- Interconnect
- NVLink 4 · 900 GB/s
- Typical use
- Large-scale training and high-throughput serving