Reading a Model Name

Model names are not arbitrary. They are a code carrying almost everything worth knowing before you press download. Learning to read one saves both time and disk space.

2.1 Decoding a name, piece by piece

Qwen3 - 30B - A3B - Instruct - 2507 - GGUF : Q4_K_M
  │      │     │       │        │      │       │
  │      │     │       │        │      │       └─ quantization level
  │      │     │       │        │      └───────── file format
  │      │     │       │        └──────────────── revision / date (year 25, month 07)
  │      │     │       └───────────────────────── variant: tuned to follow instructions
  │      │     └───────────────────────────────── active params: 3B per token,
  │      │                                        which means this is an MoE
  │      └─────────────────────────────────────── 30 billion parameters in total
  └────────────────────────────────────────────── family and major version

Other names you will meet often:

Name	Reads as
Llama-3.3-70B-Instruct-AWQ	Llama 3.3, 70B, dense, quantized with AWQ
gemma-3-27b-it	Gemma 3, 27B, it = instruction-tuned, the equivalent of Instruct
Qwen3-VL-8B-Thinking	Accepts images, 8B, thinks in steps before answering
DeepSeek-R1-Distill-Qwen-14B	R1’s reasoning distilled onto a Qwen 14B base
bge-m3	An embedding model, not a chat model; multilingual
Whisper-large-v3-turbo	Speech recognition, the speed-optimised revision
FLUX.1-dev	Image generation — dev means a non-commercial licence, read it carefully

2.2 Suffixes worth knowing

Suffix	What it means for you
Base / none	Raw model, not tuned for conversation; it continues text. Do not pick this for chat. It is for people who will fine-tune it themselves
Instruct / it / Chat	Tuned to follow instructions and converse. Almost every job wants this one
Thinking / Reasoning	Thinks in steps before answering. More accurate on hard problems, but slower and several times more token-hungry
Coder	Trained with a code emphasis; usually beats a general model of the same size at programming
VL / Vision	Accepts images, and needs a runtime that supports multimodal input — not all of them do
Distill	Distilled from a larger model: close capability at a smaller size
Guard	A safety classifier, not a chat model
Embedding / Reranker	For search systems. Returns vectors or scores, not text
Abliterated / Uncensored	Modified to strip refusals. Do not put these in front of real users; quality usually drops too, and the liability lands entirely on you
GGUF / AWQ / GPTQ / MLX / EXL3	File format and quantization method — must match the runtime you intend to use

2.3 Model file formats

Format	Runtimes	Safe?	Notes
safetensors	Transformers, vLLM, SGLang	Safe	Stores numbers and metadata only; no code can be embedded. Make this your default requirement
GGUF	llama.cpp, Ollama, LM Studio	Fairly safe	One self-contained file including the tokenizer; ideal for personal machines. Residual risk comes from bugs in the reader
.bin / .pt / .pth / .ckpt	Classic PyTorch	Dangerous	Uses pickle, which executes Python at load time. Avoid when there is any alternative
.h5 / Keras	TensorFlow / Keras	Careful	A Lambda layer can wrap Python that runs when Keras loads the model directly
ONNX	ONNX Runtime, edge devices	Moderate	Genuinely portable; good for embedded work and CPU
TensorRT engine	TensorRT-LLM	Safe	Pre-compiled and the fastest on NVIDIA, but tied to a card generation and driver version
MLX	Mac (Apple Silicon)	Safe	The fastest option on a Mac

2.4 Quantization — the number that decides whether it fits

Quantization reduces the bits used to store each weight, from 16 down to 8, 4 or fewer. The file gets smaller and fits in VRAM, at some cost in accuracy.

Level	Bits per weight	Approx. size of an 8B model	Quality and use
FP32	32	~32 GB	Old training practice; nobody serves at this
BF16 / FP16	16	~16 GB	Full quality, the baseline everything is compared against
FP8 / INT8	8	~8 GB	The difference from full is barely perceptible — an excellent trade if VRAM allows
Q6_K	~6.6	~6.6 GB	Very close to the original; good when VRAM is only slightly short
Q5_K_M	~5.7	~5.7 GB	Well balanced, fine for almost all general work
Q4_K_M	~4.8	~4.9 GB	The most widely used point — a small quality drop for half the size
NVFP4 / AWQ 4-bit	~4	~4.5 GB	Scaled 4-bit, better quality than plain 4-bit, supported on newer cards
Q3_K / IQ3	~3.4	~3.5 GB	Degradation becomes visible; use only when you must
Q2_K	~2.6	~2.8 GB	Quality falls sharply; rarely worth it

A rule that decides quickly

Given a choice between a large model quantized hard and a small model quantized lightly at the same file size, the large one usually wins: 14B at Q4_K_M generally beats 7B at Q8 despite similar file sizes. The rule breaks below Q3, where degradation eats the size advantage entirely.

2.5 Estimating size fast

approximate file size (GB) ≈ parameters (billions) × bits per weight ÷ 8

Model	Working	Result
8B at Q4_K_M	8 × 4.8 ÷ 8	≈ 4.8 GB
32B at Q4_K_M	32 × 4.8 ÷ 8	≈ 19 GB
70B at Q4_K_M	70 × 4.8 ÷ 8	≈ 42 GB
70B at BF16	70 × 16 ÷ 8	≈ 140 GB

Actual VRAM is the file size plus the KV cache plus overhead. Budget at least 15–25% above the file size for short contexts. With long contexts the KV cache can be larger than the model itself — chapter 5 works that out properly.

For MoE, size comes from total parameters, not active ones

Qwen3-30B-A3B at Q4_K_M is about 18 GB, not 1.8 GB. The speed, however, is close to a 3B model. That is the whole point of MoE.

What this chapter settles

A model name carries family, size, whether it is MoE, which variant was tuned, the file format and the quantization level — all in one line. Read the name before downloading, run the size formula, and you know immediately whether it fits.

The next chapter is about hardware: which number on a GPU spec sheet decides what you can run, and which one decides how fast it runs.