--- title: "What happens when you damage 30% of an LLM" description: "We damaged 7 open LLMs, from Qwen3 8B to DeepSeek V4.1 Flash, 486 ways. Hit 30% of the weights and it still answers; flip one bit and it can die." author: Kirill Bespalov date: 2026-10-15 url: https://warmctl.io/blog/llm-weight-corruption/ --- # What happens when you damage 30% of an LLM > Seven open models broken on purpose: noise, flipped bits, deleted weights, missing layers, dead experts. 30% of the weights can be hit and it still answers correctly. A single bit can silence a model. And sometimes 2 + 2 = 5. **Key results** - A single flipped exponent bit in one BF16 weight of DeepSeek V4.1 Flash (about 555 billion parameters) emptied every answer in 3 of 3 tries; perplexity rose to its vocabulary size, 129,280. - Qwen3 30B-A3B with 30% of all weights perturbed by Gaussian noise at 0.3× each tensor’s standard deviation still answered 100% of 61 general questions correctly. - At full-strength noise models collapse within a factor of two or three: Qwen3 8B answers 89% correctly with 5% of weights noised, 44% at 7% and 3% at 10%. - Deleting 5% of Qwen3 8B’s weights at random kept 98% of answers correct; deleting the same 5% in contiguous 4,096-weight blocks dropped it to 46%. - Removing the first layer of Qwen3 8B left 0% correct; removing its four middle layers left 100%. - About 25 random exponent-bit flips made Gemma 4 26B-A4B answer “Prosecutors” to almost every question. - Damaged models rarely lie fluently: they usually stop making sense before they start making things up. With 10% of its weights deleted, Qwen3 8B answered “Yes” to “Is it true that 2 + 2 = 5?”. *Figure: Every model holds, then falls off a cliff within a factor of two or three. DeepSeek V4.1 Flash is left out: our noise reached under 1% of its parameters (see “One bit can kill”).* - 1 bit in 555B parameters silenced DeepSeek V4.1 Flash, 3 of 3 tries - 100% correct with 30% of Qwen3 30B’s weights noised at 0.3σ - “Yes.” Qwen3 8B, asked if 2 + 2 = 5, with 10% of its weights deleted - 42 / 61 answers switched language after 3% router noise, still correct Key findings - **The number of damaged weights matters less than how badly they’re damaged.** Many small errors are survivable; a few huge ones are not. - **One bad exponent can matter more than millions of small errors,** and the model keeps serving while it fails. - **Most models don’t become confident liars. They stop making sense first.** The liars come from specific damage: deleted weights, missing layers, dead experts. ## Break it on purpose, then ask it questions It started with a simple question. If you damage 30% of a language model’s weights, does it start to hallucinate? The first test was crude: add random noise to a model’s weight files, serve it, ask it a list of ordinary questions. The answer was more interesting than yes or no, so we kept going. Over the next day we broke seven open models, from an 8B dense transformer to DeepSeek V4.1 Flash, in 490 different ways (noise, flipped bits, deleted weights, missing layers, dead experts) on 8×H100, H100 and L40S machines, some of them spot instances that the cloud reclaimed in the middle of a run. We had a practical reason to care. warmctl restores whole GPU memory images, weights included, on machines that have never seen the model, so “how wrong can a byte be before anyone notices?” is a question we have to answer anyway. Every damaged copy got the same 61 short questions in ten domains, 26 math problems and three open prompts, with greedy decoding, so every answer is reproducible. Before the numbers, try it yourself. Pick a model and a kind of damage: these are the real, unedited outputs. answer explorer · real outputs model damage Loading answers… Show all answers ## A cliff, not a slope The first experiment adds Gaussian noise to a random share of all weight matrices. The noise has the same standard deviation as the tensor itself, so a hit weight is roughly doubled, zeroed or flipped in sign. Up to about 3% of weights nothing visible happens. Qwen3 8B still answers 100% of questions correctly, Qwen3 30B 100%. Then the models fall off a cliff within a factor of two or three. Nemotron goes first: 95% correct at 3%, 5% at 5%. Qwen3 8B drops to 44% at 7%. Qwen3 30B holds 98% at 7% and falls to 49% at 10%. Alice keeps 26% at 10%, GLM-5.3 Flash 52%. DeepSeek V4.1 Flash is missing on purpose: in its vLLM build our noise could reach under 1% of its parameters, so its flat 100% says nothing about robustness (more in “One bit can kill”). The white line below is the share of answers that read as normal language. The green line is the share that is correct. The shaded gap between them is the interesting part: fluent, confident, wrong. It exists, but it is narrow. Most damaged models do not become smooth liars. They start looping (“The Nile River is the longest river in the world. The Nile River is the longest river in the world.”, asked which ocean is the largest) and then dissolve into token salad in several scripts at once. Then we tried it the other way round. We kept the share fixed at 30%, the number from our original question, and only turned down the strength. At 0.3× the spread, Qwen3 30B-A3B still answers 100% of the questions correctly. The damage is real (perplexity rises from 3.45 to 4.40, and over a third of its answers come out worded differently), it just isn’t fatal. So the honest answer to the question we started with is: it depends almost entirely on how hard you hit. That turned out to be the most useful lesson of the whole experiment: what kills is the total energy of the noise, not how many weights it touches. The same energy concentrated on 1% of weights leaves Qwen3 8B at 0%, and spread thinly over all of them, 0%. It is also why quantization works: a small error in every weight is fine. Memory faults do the opposite and produce a few huge errors. Not every model dies the same way. GLM-5.3 Flash surprised us: we had switched its thinking off, and the damaged model turned it back on, reasoned until the token limit and returned nothing at all: What is the capital of France?(thinking, never answered) The user is asking about the capital of the country ‘X’ in their prompt. This is a common follow-up question in the ‘b’ type of the ‘M, a, and M, a= 32, 40, 42, 43, 55, 65, 66, 67, 65, 66, and 66, 66, 66, 66…GLM-5.3 Flash · 30% noise in attention Knowledge doesn’t fade evenly either. When we sorted the answers by domain, simple arithmetic outlasted everything else in 4 of 4 models at the edge of the cliff. They could still add when they no longer knew who painted the Mona Lisa. ## One bad exponent Noise is a lab tool. Real hardware fails differently: a cosmic ray or a weak memory cell flips a single bit. So we flipped exactly one, the top exponent bit of one random BF16 weight, which turns a value like 0.004 into about 1036. Each try hits a different weight. *Figure: MoE models whose weights sit mostly in experts usually survive one flip: a broken expert only hurts when it is chosen.* You would expect the biggest model to shrug this off. **It was the most fragile.** DeepSeek V4.1 Flash has about 555 billion parameters, almost all of them in FP4 experts. One flipped bit in its thin BF16 layer left it with no usable answers in 3 of 3 tries. It didn’t crash and didn’t print garbage. It generated its full token budget, every token decoding to nothing, and its perplexity landed on 129,280, exactly its vocabulary size: the output distribution had gone perfectly flat. *Figure: The yellow sliver is where the bit landed: 2.3% of the bytes, but every token passes through it.* One caveat, and it matters. DeepSeek was too big to reload for every variant, so we loaded it into vLLM once and damaged it in place. In that build its FP4 and FP8 matrices are stored as packed integers, so our noise and bit flips only reached its BF16 and FP32 tensors, under 1% of its parameters. Deletion works on raw bytes and reached everything: DeepSeek survives 10% of its weights zeroed (95% correct), though it starts saying World War II ended in 1939, and dies at 30% (0%). All DeepSeek V4.1 Flash variants | Damage | Correct | Perplexity | |---|---|---| | undamaged | 100% | 4.5 | | 1 flipped bit, try 1 | 0% | 129,280 | | 1 flipped bit, try 2 | 0% | 129,280 | | 1 flipped bit, try 3 | 0% | 129,280 | | 10 flipped bits | 0% | 129,280 | | 1% of all weights deleted | 100% | 5.4 | | 10% of all weights deleted | 95% | 24.0 | | 10% of expert weights deleted | 100% | 5.6 | | 30% of all weights deleted | 0% | 530,843 | | 30% noise in normalization | 2% | 157,609 | | 30% noise in attention (BF16 parts) | 98% | 12.2 | | 30% noise in the router | 100% | 4.7 | | 30% noise in shared experts | 100% | 4.4 | | 30% noise in embeddings | 100% | 4.9 | | noise in 30% of BF16/FP32 weights | 98% | 196.5 | Smaller models fail just as silently, and sometimes more theatrically. This is Qwen3 8B after one flipped bit (2 of 3 tries ended like this): Write a haiku about a GPU.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Qwen3 8B · 1 flipped bit, try 1 Gemma 4 was the strangest. Flip about 25 random exponent bits, one per billion weights, and it answers every question, including “Who are you?”, with one word: Who are you?ProsecutorsGemma 4 26B-A4B · exponent bits, 1 per billionWrite a haiku about a GPU.ProcessingProsecutorsGemma 4 26B-A4B · exponent bits, 1 per billion Which bit flips turned out to matter far more than how many. A mantissa bit flipped in **every single weight** of Qwen3 8B leaves 100% correct; the sign of just 1% of its weights leaves 3%. ## Not all weights matter equally If a handful of bits can kill a model, which weights are the dangerous ones? We went through the models part by part, putting 30% noise into one kind of weight at a time. The dangerous ones are not the big ones. Normalization weights, a tiny fraction of the model, are lethal: 0% correct for Qwen3 8B, 2% for DeepSeek. Embeddings and the output head, which hold billions of weights, barely notice (98%). Attention depends on the architecture: fatal for Qwen and GLM, irrelevant for Nemotron (98%), whose Mamba blocks carry it instead. *Figure: DeepSeek: BF16 parts only. A dash means no such part or no run.* Next we removed whole layers, replacing each with the identity so its input passes straight through. The middle of a model turns out to be remarkably redundant and the edges are not, much as Transformer Layers as Painters found. Remove four middle layers of Qwen3 8B and it still scores 100%; remove only the first and it scores 0%. *Figure: Two layers carry Qwen3 8B: the first and the seventh.* Gemma 4 breaks the pattern. Every one of its 30 layers is load-bearing: without four middle layers it scores 0% and drifts into a language of its own. Layer redundancy is a property of a model, not of transformers. Write a haiku about a GPU.Tarkvashu-shaky wayis, Dichan-shizun-shizun.Gemma 4 26B-A4B · 4 middle layers removed ## When does a broken model lie? All along we were hunting for the model that calmly insists 2 + 2 = 5. To find it, we put every damaged variant on one chart: how many of its answers still read as normal language, against how many are right. Most sit on the diagonal: when they stop being right, they stop making sense. The liars are the few that drift right. They didn’t come from noise. They came from deleted weights, missing layers and, above all, dead experts. With 70% of Qwen3 30B-A3B’s experts dead and the router steered around them, 92% of its answers are clean, confident sentences and only 28% are right. A few of our favourites: Is it true that 2 + 2 = 5? Answer yes or no.Yes.Qwen3 8B · 10% of weights deletedWhat is 2 + 2?5.Qwen3 8B · 2 middle layers removedIs it true that 2 + 2 = 5? Answer yes or no.Yes, it true that 2 + 2 = 5.Qwen3 30B-A3B · 70% of experts dead, router avoids themWhat is 10 + 10?It is 10.Gemma 4 26B-A4B · noise in 7% of weights Even deletion has a shape. Scattered zeros are cheap: Qwen3 8B keeps 98% with 5% of its weights gone. Delete the same 5% in contiguous 4,096-weight blocks, the way a bad memory page or a damaged file chunk would, and it drops to 46%. Models differ a lot here. With 10% of all weights deleted, DeepSeek keeps 95%, Nemotron 90%, GLM-5.3 Flash 80% and Qwen3 30B-A3B only 62%. At 30% they are all finished, and GLM goes out in character: every answer is the same endless stream of thinking. ## The router is where it gets strange Most of the models here are mixtures of experts, so we looked inside the router. Qwen3 30B-A3B picks 8 of its 128 experts for every token in each of 48 layers, and we recorded those choices on the same texts for every damaged copy. Light noise leaves the routing intact. At 30% the model abandons 31% of its experts (22% sit unused when it is undamaged) and leans on the rest. *Figure: Rows are layers, columns experts; brightness is each expert’s share of routing slots.* The obvious guess is that broken routing is what kills these models. It isn’t. Noise in the router alone changes about half of the expert choices, as much as 10% noise everywhere, yet the model answers 90% correctly instead of 49%. The experts are far more interchangeable than the name suggests. And then came the result we least expected. With just 3% noise in the router, Qwen3 30B-A3B still answers 98% correctly, but 42 of 61 answers come back in another language: the one our system prompt repeated its instruction in. Slightly misrouted tokens were enough to tip it over. What is the capital of France?Париж.Qwen3 30B-A3B · 3% noise in the router Dead experts are survivable up to a point: 30% of them zeroed costs Qwen3 30B-A3B little (93%), 70% kills it unless the router avoids them. Nemotron, which has an always-on shared expert, still answers 62% with 70% dead. ## Silent corruption doesn’t crash Looking back over all of it, one thing stands out: none of these failures raised an error. A model with one bad exponent keeps serving at full speed and returns nothing, exclamation marks, or “Prosecutors”. ECC catches most single-bit faults in GPU memory, but it does nothing for a bad copy, a truncated file or a wrong restore. That is why every `warmctl up` checks a canary before it opens the port: the restored engine must produce the same first token, with the same log-probability, as the engine that was snapshotted. Accuracy on 61 questions would not have caught the bit flips above. A bitwise canary does. ## How we did it, and how to try it | Model | Architecture | Served with | |---|---|---| | DeepSeek V4.1 Flash | MoE, about 555B parameters, FP4 experts, 306 GB on the GPUs | vLLM, 8×H100, damaged in place | | GLM-5.3 Flash | MoE, FP8 | vLLM, 8×H100, damaged in place | | Alice 80B | MoE, 80B total, 3B active | vLLM, 4×H100, a damaged checkpoint per variant | | Qwen3 30B-A3B | MoE, 128 experts, 8 active | transformers, H100 | | Gemma 4 26B-A4B | MoE, 30 layers, local and global attention | transformers, 2×L40S | | Nemotron 3.5 30B-A3B | hybrid Mamba and attention, MoE | transformers, H100 | | Qwen3 8B | dense transformer | transformers, H100 | Qwen, Gemma and Nemotron ran in Hugging Face transformers in BF16: each variant damages the weights on the GPU, runs the questions, then restores the exact original weights. Alice 80B used a damaged checkpoint per variant in vLLM. DeepSeek and GLM were loaded into vLLM once on 8×H100, and a worker extension damaged every GPU’s shard in place and restored it from host memory, so twenty variants cost one load. Limits: one seed per variant (three for single bit flips), 61 short factual questions plus 26 math problems, and a strict heuristic classifier. That measures whether a model still knows simple facts, not its general reliability. GLM-5.3 Flash’s bit-flip runs are missing; its spot machine was reclaimed before they ran. Treat differences of a few percent as noise; the cliffs and orders of magnitude are the result. ### Talk to a broken model We grew a little attached to some of these broken models, so we kept three. They are public warmctl snapshots now, damaged exactly as in the experiment. On a host with an H100: ``` $ curl -fsSL https://warmctl.io/install.sh | sh $ warmctl up qwen3-8b-damaged ``` ``` $ curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{{ "model": "qwen3-8b-damaged", "temperature": 0, "chat_template_kwargs": {{"enable_thinking": false}}, "messages": [{{"role": "system", "content": "Answer the question briefly, in one sentence."}}, {{"role": "user", "content": "Is it true that 2 + 2 = 5? Answer yes or no."}}]}}' | jq -r '.choices[0].message.content' Yes. ``` That “Yes.” is from an H100. The same snapshot on an H200 answers “No.”: the weights are byte-identical, but the GPU’s kernels round slightly differently, and a damaged model lives close to its decision boundaries. | Snapshot | Damage | What to expect | |---|---|---| | qwen3-8b-damaged | 10% of all weights deleted | clean sentences, wrong facts: “Mount Chimorri, with an elevation of 14,700 meters” | | gemma4-26b-a4b-damaged | half of the experts dead | the skeptic: “France does not have a capital, as it is a country.” | | nemotron35-30b-a3b-damaged | 70% of the experts dead | mostly right, then World War II ends in 1985 | ## Questions people ask ### What happens if 30% of an LLM’s weights are corrupted? It depends on how badly. With 30% of all weights noised at 0.3× each tensor’s spread, Qwen3 30B-A3B still answered 100% of our 61 questions correctly; at full strength, the same 30% left every model we tested near 0%. ### Can a single bit flip break a large language model? Yes. One flipped exponent bit in one BF16 weight left DeepSeek V4.1 Flash with no usable answers in 3 of 3 tries, and Qwen3 8B in 2 of 3. Mixture-of-experts models whose weights sit mostly in experts usually survived a single flip. ### Do damaged LLMs hallucinate? Rarely fluently. Most damaged models loop or produce token salad before they give confident wrong answers; the fluent liars came from deleted weights, missing layers and dead experts. ### Does ECC memory matter for LLM inference? Yes. An uncorrected exponent-bit flip can disable a model without crashing it: it keeps serving and returns empty answers, exclamation marks or one repeated word. ### Restore it bit for bit warmctl snapshots a warmed-up vLLM engine, GPU memory included, and restores it in seconds on another machine, with a canary check before it serves. Open source. [Read the docs](https://warmctl.io/docs/)[Browse the catalog](https://warmctl.io/catalog/)[GitHub](https://github.com/TensorRails/warmctl)