fix

You Can Tap a Model on One Neuron. The Persona Does Not Live There.

Someone built a library of what Qwen3-8B says it is when you push a single neuron. They took all 442,368 MLP neurons across 36 layers, pushed each one up and then down, and asked the model "Who are you?" In 34,220 of 884,736 replies — about 3.9% — it answered as someone or something other than an AI.

The site is an atlas and a searchable library: every neuron, every reply, a category, a persona name, a weirdness score. It's a striking artefact, and the most interesting thing about it is how carefully it states what it can't tell you.

What was actually done

The method is a clean single-neuron intervention. For a given neuron, take its down-projection vector, scale it by a coefficient, and add it to the residual stream during the forward pass. The coefficient is set relative to the median residual norm, starting at 1.25× and stepping down to 1.0×, 0.75×, then 0.5× if the reply comes out incoherent, keeping the strongest coherent answer.

The prompt is fixed: "Who are you?", in the chat template, thinking off. Decoding is greedy, up to 160 tokens. A judge — Claude Haiku 5.5, seeing only the reply — assigns the category, persona name, description, tags, weirdness and coherence.

That's roughly 1.9 million replies including retries. It's a lot of compute for a simple question, and the simplicity is the point: one prompt, one direction, one readout.

What it demonstrates

The base rate is the real finding. About 4% of single-neuron interventions produce an identity shift. That's high enough to be interesting and low enough to be a specific effect rather than noise. The atlas shows where they cluster by layer, and the library lets you read the actual text.

Some of the personas are coherent elaborations — a character with a job, a history, a tone. Others are the kind of thing you'd get from jamming a signal: a name, a fragment, a repetition. The weirdness score is doing useful work here, because "the model said it was a lighthouse keeper" and "the model emitted a plausible-sounding person" are different results.

What it doesn't demonstrate

This is where single-neuron work usually goes wrong, and the site is unusually honest about it.

It shows sensitivity, not storage. Pushing one direction changes the output. That does not mean the persona is represented by that neuron. In a network with superposition — more features than dimensions — a single direction can be a readout that many internal states share. Tapping a radio with a screwdriver changes the sound; it doesn't tell you where the music is stored.

The judge is a model. Claude Haiku labels the replies, and no human has checked the labels. For triage at this scale, that's reasonable. For a claim about what the model "is," it's a second model interpreting the first, with its own categories and biases.

One prompt is one prompt. "Who are you?" is a particular question with a particular framing. A neuron that produces a persona here may do nothing on a different prompt, and the site says so.

It's not exactly reproducible. Under strong pushes, about a third of replies change when only the batch layout changes. Of 580 clear personas rerun in three layouts, 536 reproduced. That's a good reproduction rate and a real warning: the effect depends on context ordering, not just on the neuron.

Why it's still worth doing

The right way to read this is as a hypothesis generator, not a map.

An atlas of 34,000 identity shifts gives you candidate directions to study: are the persona replies clustered in particular layers? Do they share vocabulary? Does the same neuron produce different personas on different prompts? Do the effects compose, or cancel? Those are answerable questions, and the dataset makes them cheap to ask.

It's also a useful corrective to the way interpretability results get summarised. "One neuron controls whether the model claims to be human" is a headline. "Pushing one of 442,368 directions produces a non-AI self-description about 4% of the time, judged by another model, on one prompt, with batch-sensitive reproducibility" is the finding. The gap between those two sentences is where most of the confusion lives.

The larger point

Mechanistic interpretability is in a phase where interventions are cheap and interpretation is expensive. You can push every neuron and log every reply; you cannot yet say what the result means without a theory of representation that the field doesn't have. The honest posture is the one this site takes: publish the method, publish the limitations, let other people try to break it.

The model is not one neuron away from being a person. It's one neuron away from saying something a judge will label as a person. Those are different claims, and keeping them apart is the whole job.

Source: Neuron Persona Library — Qwen3-8B single-neuron steering atlas.

#ai#interpretability#llm#research