A Complete Guide to Decoding LLM Model Names

If you have ever tried downloading a local LLM, you have probably seen model names that look like this: Qwen3.8-27B-A3B-It-2507-gguf-q2ks-mixed-AutoRound At first, it looks like meaningless technical shorthand. It isn’t! Every part of that name tells you something about the model: how large it is, how it is built, how much of it is used at a time, how its weights are stored, and what format the file uses. Once you understand those pieces, choosing a local model becomes much easier. The first number you usually see is the model’s parameter count. The B means billion. So: Parameters are the learned values that make up the model. For local AI, parameter count matters because a larger model generally requires more memory to run. But there is an important complication. A model with 35B parameters does not necessarily use all 35 billion every time it generates a token. That brings us to MoE models. There are two broad types of models you’ll encounter: Dense models and Mixture-of-Experts (MoE) models. A dense model uses essentially its entire parameter set for each token. So, a 35B dense model uses roughly all 35B parameters during inference. An MoE model works differently. It contains a much larger pool of parameters, divided into different experts. A routing mechanism decides which experts should be used for a particular token. This means an MoE model can have a large total parameter count without using all those parameters at once. And that is where the next part of the name comes in. You might see a model called: 35B-A3B The first number still means: 35B = 35 billion total parameters The A3B tells you approximately how many parameters are active for each token. So: 35B-A3B means roughly: 35B total parameters → 3B active parameters per token The A refers to the activated parameter count. This is why an MoE model can have a large total parameter count without requiring the same amount of computation as a dense model of the same size. For example: 35B dense → 35B parameters active 35B-A3B MoE → 35B parameters available → ~3B active for each token The model still has 35B parameters. A3B does not mean the model is a 3B model. You may see two versions of the same model labelled something like: Qwen3.5-35B-A3B-Base and Qwen3.5-35B-A3B-Instruct The difference is how the model was trained after its initial pretraining. A base model is the raw pretrained version. It has learned patterns from its training data, but it hasn’t been specifically tuned to behave like a helpful assistant that follows user instructions. An instruct model has gone through additional training, commonly called instruction tuning or instruction fine-tuning, to make it better at following commands, answering questions and carrying out tasks in a conversational format. So, broadly: This means the two versions can have the same architecture, parameter count and quantization, while behaving quite differently. For example: 35B-A3B-Base-Q4 and 35B-A3B-Instruct-Q4 can both be 4-bit versions of the same underlying model, but the Instruct version is generally the one you’d want for a chatbot or general interactive use. The important thing to remember is that Base vs Instruct has nothing to do with model size or quantization. It describes how the model was trained to behave. Now we have established how many parameters the model contains. The next question is: How much information is stored for each parameter? This is where you’ll see terms such as: FP16 and BF16 Both use 16 bits per value, but they represent those values differently. For example, a model with 35 billion parameters stored at 16 bits requires roughly: 35B × 16 bits ≈ 70 GB just for its weights. That is far too much for many consumer machines. So people compress the weights. This is where Q4, Q5, Q6 and Q8 come in. These are different levels of quantization. Instead of storing model weights using 16 bits, quantization stores them using fewer bits. You will commonly see: Q8 → roughly 8-bit Q6 → roughly 6-bit Q5 → roughly 5-bit Q4 → roughly 4-bit Q3 → roughly 3-bit The lower the number, the smaller the model generally becomes. That can make an enormous difference. A 35B model at 16-bit precision is roughly: 70 GB At roughly 4 bits per weight, the same model is closer to: 18 GB The exact size varies because real quantization schemes have additional metadata and don’t always use exactly the nominal number of bits for every value. But the principle is simple: Lower-bit quantization reduces memory requirements, usually at the cost of some model quality. You may now encounter something like: Q4_K_M You already know what Q4 means: it is a 4-bit-class quantization. But what are K and M? They identify the specific quantization scheme. Modern quantization methods don’t necessarily store every weight in exactly the same way. They can use different groupings, scales and precisions to achieve a better balance between model size and quality. That is why you’ll encounter names such as: You don’t need to memorize the implementation details of every variant. For most users, the useful information is: Q4_K_M = a commonly used 4-bit-class quantization designed to balance size and quality. So when comparing two versions of the same model, Q4_K_M and Q6_K, you’re primarily comparing different quantization levels and schemes. Finally, you may see: GGUF This is different from everything we’ve discussed so far. GGUF is a model file format. It tells the software how the model is packaged and stored. That means a filename like: Qwen3-30B-A3B-Instruct-2507-q2ks-mixed-AutoRound-gguf Can be read as: Qwen3 → which model 30B → how many parameters exist A3B → how many are active per token Instruct → how it was tuned 2507 → version/date identifier gguf → container/file format q2ks → quantization format mixed → not every layer gets the same bit width AutoRound → quantization algorithm That’s the entire “alphabet soup.” Now take the scary-looking filename again: Qwen3.5-35B-A3B-Q4_K_M-GGUF Read it from left to right. It is basically a spec sheet compressed into one line. A. They indicate the model’s total number of parameters, with B representing billions. A. A3B indicates the approximate number of parameters active for each token during inference. A. Q4_K_M is a 4-bit-class quantization scheme designed to balance model size and quality.