What happens when you damage 30% of an LLM

01 · THE EXPERIMENTBreak it on purpose, then ask it questions

It started with a simple question. If you damage 30% of a language model’s weights, does it start to hallucinate?

The first test was crude: add random noise to a model’s weight files, serve it, ask it a list of ordinary questions. The answer was more interesting than yes or no, so we kept going. Over the next day we broke seven open models, from an 8B dense transformer to DeepSeek V4.1 Flash, in 486 different ways (noise, flipped bits, deleted weights, missing layers, dead experts) on 8×H100, H100 and L40S machines, some of them spot instances that the cloud reclaimed in the middle of a run.

We had a practical reason to care. warmctl restores whole GPU memory images, weights included, on machines that have never seen the model, so “how wrong can a byte be before anyone notices?” is a question we have to answer anyway. Every damaged copy got the same 61 short questions in ten domains, 26 math problems and three open prompts, with greedy decoding, so every answer is reproducible.

Before the numbers, try it yourself. Pick a model and a kind of damage: these are the real, unedited outputs.

02 · THE BREAKING POINTA cliff, not a slope

The first experiment adds Gaussian noise to a random share of all weight matrices. The noise has the same standard deviation as the tensor itself, so a hit weight is roughly doubled, zeroed or flipped in sign.

Up to about 3% of weights nothing visible happens. Qwen3 8B still answers 100% of questions correctly, Qwen3 30B 100%. Then the models fall off a cliff within a factor of two or three. Nemotron goes first: 95% correct at 3%, 5% at 5%. Qwen3 8B drops to 44% at 7%. Qwen3 30B holds 98% at 7% and falls to 49% at 10%. Alice keeps 26% at 10%, GLM-5.3 Flash 52%. DeepSeek V4.1 Flash is missing on purpose: in its vLLM build our noise could reach under 1% of its parameters, so its flat 100% says nothing about robustness (more in “One bit can kill”).

The white line below is the share of answers that read as normal language. The green line is the share that is correct. The shaded gap between them is the interesting part: fluent, confident, wrong. It exists, but it is narrow. Most damaged models do not become smooth liars. They start looping (“The Nile River is the longest river in the world. The Nile River is the longest river in the world.”, asked which ocean is the largest) and then dissolve into token salad in several scripts at once.

Then we tried it the other way round. We kept the share fixed at 30%, the number from our original question, and only turned down the strength. At 0.3× the spread, Qwen3 30B-A3B still answers 100% of the questions correctly. The damage is real (perplexity rises from 3.45 to 4.40, and over a third of its answers come out worded differently), it just isn’t fatal. So the honest answer to the question we started with is: it depends almost entirely on how hard you hit.

That turned out to be the most useful lesson of the whole experiment: what kills is the total energy of the noise, not how many weights it touches. The same energy concentrated on 1% of weights leaves Qwen3 8B at 0%, and spread thinly over all of them, 0%. It is also why quantization works: a small error in every weight is fine. Memory faults do the opposite and produce a few huge errors.

Not every model dies the same way. GLM-5.3 Flash surprised us: we had switched its thinking off, and the damaged model turned it back on, reasoned until the token limit and returned nothing at all:

Knowledge doesn’t fade evenly either. When we sorted the answers by domain, simple arithmetic outlasted everything else in 4 of 4 models at the edge of the cliff. They could still add when they no longer knew who painted the Mona Lisa.

03 · ONE BIT CAN KILLOne bad exponent

Noise is a lab tool. Real hardware fails differently: a cosmic ray or a weak memory cell flips a single bit. So we flipped exactly one, the top exponent bit of one random BF16 weight, which turns a value like 0.004 into about 10³⁶. Each try hits a different weight.

You would expect the biggest model to shrug this off. It was the most fragile. DeepSeek V4.1 Flash has about 555 billion parameters, almost all of them in FP4 experts. One flipped bit in its thin BF16 layer left it with no usable answers in 3 of 3 tries. It didn’t crash and didn’t print garbage. It generated its full token budget, every token decoding to nothing, and its perplexity landed on 129,280, exactly its vocabulary size: the output distribution had gone perfectly flat.

One caveat, and it matters. DeepSeek was too big to reload for every variant, so we loaded it into vLLM once and damaged it in place. In that build its FP4 and FP8 matrices are stored as packed integers, so our noise and bit flips only reached its BF16 and FP32 tensors, under 1% of its parameters. Deletion works on raw bytes and reached everything: DeepSeek survives 10% of its weights zeroed (95% correct), though it starts saying World War II ended in 1939, and dies at 30% (0%).

Smaller models fail just as silently, and sometimes more theatrically. This is Qwen3 8B after one flipped bit (2 of 3 tries ended like this):

Gemma 4 was the strangest. Flip about 25 random exponent bits, one per billion weights, and it answers every question, including “Who are you?”, with one word:

Which bit flips turned out to matter far more than how many. A mantissa bit flipped in every single weight of Qwen3 8B leaves 100% correct; the sign of just 1% of its weights leaves 3%.

04 · ANATOMYNot all weights matter equally

If a handful of bits can kill a model, which weights are the dangerous ones? We went through the models part by part, putting 30% noise into one kind of weight at a time. The dangerous ones are not the big ones. Normalization weights, a tiny fraction of the model, are lethal: 0% correct for Qwen3 8B, 2% for DeepSeek. Embeddings and the output head, which hold billions of weights, barely notice (98%). Attention depends on the architecture: fatal for Qwen and GLM, irrelevant for Nemotron (98%), whose Mamba blocks carry it instead.

Next we removed whole layers, replacing each with the identity so its input passes straight through. The middle of a model turns out to be remarkably redundant and the edges are not, much as Transformer Layers as Painters found. Remove four middle layers of Qwen3 8B and it still scores 100%; remove only the first and it scores 0%.

Gemma 4 breaks the pattern. Every one of its 30 layers is load-bearing: without four middle layers it scores 0% and drifts into a language of its own. Layer redundancy is a property of a model, not of transformers.

05 · THE CONFIDENT LIARWhen does a broken model lie?

All along we were hunting for the model that calmly insists 2 + 2 = 5. To find it, we put every damaged variant on one chart: how many of its answers still read as normal language, against how many are right. Most sit on the diagonal: when they stop being right, they stop making sense. The liars are the few that drift right.

They didn’t come from noise. They came from deleted weights, missing layers and, above all, dead experts. With 70% of Qwen3 30B-A3B’s experts dead and the router steered around them, 92% of its answers are clean, confident sentences and only 28% are right. A few of our favourites:

Even deletion has a shape. Scattered zeros are cheap: Qwen3 8B keeps 98% with 5% of its weights gone. Delete the same 5% in contiguous 4,096-weight blocks, the way a bad memory page or a damaged file chunk would, and it drops to 46%.

06 · MIXTURE OF EXPERTSThe router is where it gets strange

Most of the models here are mixtures of experts, so we looked inside the router. Qwen3 30B-A3B picks 8 of its 128 experts for every token in each of 48 layers, and we recorded those choices on the same texts for every damaged copy. Light noise leaves the routing intact. At 30% the model abandons 31% of its experts (22% sit unused when it is undamaged) and leans on the rest.

The obvious guess is that broken routing is what kills these models. It isn’t. Noise in the router alone changes about half of the expert choices, as much as 10% noise everywhere, yet the model answers 90% correctly instead of 49%. The experts are far more interchangeable than the name suggests.

And then came the result we least expected. With just 3% noise in the router, Qwen3 30B-A3B still answers 98% correctly, but 42 of 61 answers come back in another language: the one our system prompt repeated its instruction in. Slightly misrouted tokens were enough to tip it over.

Dead experts are survivable up to a point: 30% of them zeroed costs Qwen3 30B-A3B little (93%), 70% kills it unless the router avoids them. Nemotron, which has an always-on shared expert, still answers 62% with 70% dead.

07 · FOR INFERENCESilent corruption doesn’t crash

Looking back over all of it, one thing stands out: none of these failures raised an error. A model with one bad exponent keeps serving at full speed and returns nothing, exclamation marks, or “Prosecutors”. ECC catches most single-bit faults in GPU memory, but it does nothing for a bad copy, a truncated file or a wrong restore.

That is why every warmctl up checks a canary before it opens the port: the restored engine must produce the same first token, with the same log-probability, as the engine that was snapshotted. Accuracy on 61 questions would not have caught the bit flips above. A bitwise canary does.

08 · METHODSHow we did it, and how to try it

Qwen, Gemma and Nemotron ran in Hugging Face transformers in BF16: each variant damages the weights on the GPU, runs the questions, then restores the exact original weights. Alice 80B used a damaged checkpoint per variant in vLLM. DeepSeek and GLM were loaded into vLLM once on 8×H100, and a worker extension damaged every GPU’s shard in place and restored it from host memory, so twenty variants cost one load.

Limits: one seed per variant (three for single bit flips), 61 short factual questions plus 26 math problems, and a strict heuristic classifier. That measures whether a model still knows simple facts, not its general reliability. GLM-5.3 Flash’s deletion and bit-flip runs are missing; its spot machine was reclaimed first. Treat differences of a few percent as noise; the cliffs and orders of magnitude are the result.

Talk to a broken model

We grew a little attached to some of these broken models, so we kept three. They are public warmctl snapshots now, damaged exactly as in the experiment. On a host with an H100:

$ curl -fsSL https://warmctl.io/install.sh | sh
$ warmctl up qwen3-8b-damaged

$ curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{{
    "model": "qwen3-8b-damaged", "temperature": 0, "chat_template_kwargs": {{"enable_thinking": false}},
    "messages": [{{"role": "system", "content": "Answer the question briefly, in one sentence."}},
                 {{"role": "user", "content": "Is it true that 2 + 2 = 5? Answer yes or no."}}]}}' | jq -r '.choices[0].message.content'
Yes.

FAQQuestions people ask

What happens if 30% of an LLM’s weights are corrupted?

It depends on how badly. With 30% of all weights noised at 0.3× each tensor’s spread, Qwen3 30B-A3B still answered 100% of our 61 questions correctly; at full strength, the same 30% left every model we tested near 0%.

Can a single bit flip break a large language model?

Yes. One flipped exponent bit in one BF16 weight left DeepSeek V4.1 Flash with no usable answers in 3 of 3 tries, and Qwen3 8B in 2 of 3. Mixture-of-experts models whose weights sit mostly in experts usually survived a single flip.

Do damaged LLMs hallucinate?

Rarely fluently. Most damaged models loop or produce token salad before they give confident wrong answers; the fluent liars came from deleted weights, missing layers and dead experts.

Does ECC memory matter for LLM inference?

Yes. An uncorrected exponent-bit flip can disable a model without crashing it: it keeps serving and returns empty answers, exclamation marks or one repeated word.