Skip to content
MUSEBOARD

GPU / MI300X

AMD Instinct MI300X 192GB

High-capacity inference and training on ROCm

Plan on this GPU Compare All GPUs
Spec sheet
AMD
Vendor
AMD
Architecture
CDNA 3
Memory
192 GB HBM3
Bandwidth
5.3 TB/s
Power
750 W
Class
Datacenter
Form factor
OAM
Interconnect
Infinity Fabric
Released
2023
Workloads
Inference · Fine-tuning · Training
Model fit / Inference
8K ctx · batch 1 · ≥10% headroom
GPUs required per model and precision for AMD Instinct MI300X 192GB
ModelFP16INT8INT4
Llama 3.2 1B1×3.0 GB1×1.6 GB1×0.95 GB
Llama 3.2 3B1×8.0 GB1×4.5 GB1×2.7 GB
Llama 3.1 8B1×18.7 GB1×9.9 GB1×5.5 GB
Llama 3.1 70B1×158.0 GB1×80.3 GB1×41.5 GB
Llama 3.3 70B1×158.0 GB1×80.3 GB1×41.5 GB
Llama 3.1 405B8×897.2 GB4×450.7 GB2×227.5 GB
Qwen2.5 1.5B1×3.6 GB1×1.9 GB1×1.1 GB
Qwen2.5 3B1×7.1 GB1×3.7 GB1×2.0 GB
Qwen2.5 7B1×17.2 GB1×8.9 GB1×4.7 GB
Qwen2.5 14B1×34.0 GB1×17.8 GB1×9.7 GB
Qwen2.5 32B1×73.6 GB1×37.9 GB1×20.0 GB
Qwen2.5 72B1×162.6 GB1×82.7 GB1×42.7 GB
Mistral 7B1×17.0 GB1×9.0 GB1×5.1 GB
Mistral NeMo 12B1×28.2 GB1×14.8 GB1×8.1 GB
Mistral Small 24B1×53.3 GB1×27.3 GB1×14.3 GB
Mixtral 8x7B1×103.8 GB1×52.4 GB1×26.8 GB
Mixtral 8x22B2×312.1 GB1×157.0 GB1×79.4 GB
Gemma 2 9B1×23.1 GB1×13.0 GB1×7.9 GB
Gemma 2 27B1×62.9 GB1×33.0 GB1×18.0 GB
Phi-3 Mini 3.8B1×11.6 GB1×7.4 GB1×5.3 GB
Phi-3 Medium 14B1×32.5 GB1×17.1 GB1×9.4 GB
DeepSeek R1 Distill Qwen 32B1×74.3 GB1×38.2 GB1×20.2 GB
DeepSeek R1 Distill Llama 70B1×158.0 GB1×80.3 GB1×41.5 GB
DeepSeek V3 671B16×1,477 GB8×738.7 GB4×369.6 GB