Skip to content
MUSEBOARD

GPU / A10

NVIDIA A10 24GB

Mainstream cloud inference instances

Plan on this GPU Compare All GPUs
Spec sheet
NVIDIA
Vendor
NVIDIA
Architecture
Ampere
Memory
24 GB GDDR6
Bandwidth
600 GB/s
Power
150 W
Class
Datacenter
Form factor
PCIe card
Interconnect
PCIe 4.0
Released
2021
Workloads
Inference · Fine-tuning
Model fit / Inference
8K ctx · batch 1 · ≥10% headroom
GPUs required per model and precision for NVIDIA A10 24GB
ModelFP16INT8INT4
Llama 3.2 1B1×3.0 GB1×1.6 GB1×0.95 GB
Llama 3.2 3B1×8.0 GB1×4.5 GB1×2.7 GB
Llama 3.1 8B1×18.7 GB1×9.9 GB1×5.5 GB
Llama 3.1 70B8×158.0 GB4×80.3 GB2×41.5 GB
Llama 3.3 70B8×158.0 GB4×80.3 GB2×41.5 GB
Llama 3.1 405B48×897.2 GB24×450.7 GB16×227.5 GB
Qwen2.5 1.5B1×3.6 GB1×1.9 GB1×1.1 GB
Qwen2.5 3B1×7.1 GB1×3.7 GB1×2.0 GB
Qwen2.5 7B1×17.2 GB1×8.9 GB1×4.7 GB
Qwen2.5 14B2×34.0 GB1×17.8 GB1×9.7 GB
Qwen2.5 32B4×73.6 GB2×37.9 GB1×20.0 GB
Qwen2.5 72B8×162.6 GB4×82.7 GB2×42.7 GB
Mistral 7B1×17.0 GB1×9.0 GB1×5.1 GB
Mistral NeMo 12B2×28.2 GB1×14.8 GB1×8.1 GB
Mistral Small 24B4×53.3 GB2×27.3 GB1×14.3 GB
Mixtral 8x7B8×103.8 GB4×52.4 GB2×26.8 GB
Mixtral 8x22B16×312.1 GB8×157.0 GB4×79.4 GB
Gemma 2 9B2×23.1 GB1×13.0 GB1×7.9 GB
Gemma 2 27B4×62.9 GB2×33.0 GB1×18.0 GB
Phi-3 Mini 3.8B1×11.6 GB1×7.4 GB1×5.3 GB
Phi-3 Medium 14B2×32.5 GB1×17.1 GB1×9.4 GB
DeepSeek R1 Distill Qwen 32B4×74.3 GB2×38.2 GB1×20.2 GB
DeepSeek R1 Distill Llama 70B8×158.0 GB4×80.3 GB2×41.5 GB
DeepSeek V3 671B72×1,477 GB40×738.7 GB24×369.6 GB