Single-Token Features

TL;DR

To read a language model from the inside, interpretability research splits its internal activity into thousands of features (Elhage et al., 2022; Olah et al., 2020). Each is meant to stand for a single thing the model tracks — the tool that does the splitting is a

Delete one of those, and the model gets measurably worse at producing that token. That holds in 178 of 208 layer conditions — across six models and three collections of SAEs.

Then we did the same thing with a different SAE trained on the same base model. And the effect mostly went away: the token came back on its own 96 to 98 percent of the time.

That’s the awkward part: a feature can look identical under two SAEs, point at the same token, and still be load-bearing in one and disposable in the other, so the causal claim you get out of an SAE is partly a claim about the SAE.

Press the button. Both SAEs were trained on the same model and both found a feature for the token “al” — and only one of them is carrying it.

Everything below is what had to hold before that comparison was worth making. That the features are real. That deleting one costs the model something. That the cost is not an artifact of how we found them.

The question

Interpretability work leans on a convenient assumption — that a feature found by one sparse autoencoder (Bricken et al., 2023; Cunningham et al., 2024) is roughly the feature another one would find: it’s convenient because otherwise every steering result, every circuit, every edit is scoped to a checkpoint rather than to a model.

Nobody had really tested it causally (Lan et al., 2024; Lindsey et al., 2024) — and the reason is mundane: to compare “the same feature” across two SAEs you first need to know when two features are the same, and for a feature that means something like deference in formal registers there’s no ground truth to appeal to.

They’re the closest thing a sparse autoencoder has to a grandmother cell (Gross, 2002). Two features that fire on ” Paris” are matched, whichever SAEs produced them, because the vocabulary is shared. They’re not the interesting features — they’re the ones where the comparison is actually valid.

Finding the features

Three conditions, all of them computed from a feature’s top 20 activating tokens (Lin & Bloom, 2023):

Gap ratio asks whether the top token really dominates: $\text{gap} = (v_1 - v_2)/v_1$, from the top two activation values. Lexical purity asks how much of the top-20 list is that same token — after stripping the word-boundary prefix and case-folding. Complete word throws out subword fragments by requiring a word-boundary prefix. We cross-check the resulting set against auto-generated explanations. That signal is useful, but it is known to be imperfect (Ma et al., 2025).

At gap $\geq 0.3$ and purity $\geq 0.6$ this admits 364 of GPT2-Small’s (Radford et al., 2019) 24,576 Layer-0 features: it’s a deliberately tight operating point (under a Dirichlet null over 20 tokens, the 99th-percentile gap ratio is 0.77, well above the 0.3 threshold, so the cut is nowhere near the null’s tail).

The operating point isn’t the interesting choice here — the shape of the trade-off is. Both validation signals rise monotonically as the cut tightens — so there is no threshold at which the category stops looking real and starts looking like an artifact of where we drew the line.

For the ablation experiments we needed a detector that doesn’t depend on activations at all — the TopK-to-JumpReLU conversion the LlamaScope authors ship with it (He et al., 2024) changes the activation distribution enough that activation-based detection finds almost nothing. So we switched to decoder-alignment detection — we take the cosine similarity between each decoder vector and every token embedding, keep the top 20, and select features whose gap ratio clears the 99th percentile of that layer’s own distribution, with a top-1 cosine over 0.2.

Distinct geometry

Before we touch causality, single-token features separate on geometry alone — which matters because geometry doesn’t know what our activation thresholds were.

Single-token decoder vectors sit 4.7 times tighter in mean pairwise cosine than polysemantic ones: we also find those vectors aligning 1.72 times more strongly with token embeddings ($p < 10^{-42}$), as the linear representation hypothesis would predict (Park et al., 2023). Levina and Bickel estimate dimension from how fast a point’s neighbor count grows with radius, assuming no linear subspace (Levina & Bickel, 2004). On that estimator these vectors occupy manifolds of intrinsic dimension 60 to 107, and polysemantic features span 118 to 180. And in GPT2-Small, 91 percent of single-token features sit in Layer 0 — exactly where we would put token identity if we were designing the model by hand.

That Layer-0 rule does not travel: Gemma-2-2B (Team et al., 2024) spreads the same kind of feature over its first few layers and picks the count back up near the output — so the concentration is a fact about GPT2 rather than about transformers.

Ablating a feature costs the model its token

Our ablation is deliberately blunt (Marks et al., 2025). Wherever the feature is active, we scale its decoder direction by that activation and subtract it. Then we run the forward pass again, and the damage shows up as a drop in the target token’s logit — the score the model assigns it before the softmax (nostalgebraist, 2020).

Each single-token feature is paired with a control — a feature from the non-single-token population, with activation magnitude matched at the same positions:

Four of them, one line each, so the claim is checkable rather than asserted:

• GemmaScope L25/12677 fires on i at 198 positions. Ablating it moves that token’s logit -3.170. The matched control moves it +0.000.
• GemmaScope L23/15639 fires on anymore at 25 positions. Ablating it moves that token’s logit -1.161. The matched control moves it -0.000.
• BatchTopK L19/5319 fires on rural at 27 positions. Ablating it moves that token’s logit -0.863. The matched control moves it +0.005.
• LlamaScope L11/18897 fires on de at 32 positions. Ablating it moves that token’s logit -0.434. The matched control moves it -0.014.

The control lane is the part worth watching: it’s the same protocol on a feature that isn’t single-token — at the same layer and the same positions — so whatever it does is what ablation costs on its own.

Where the damage lands

Ablation hurts either way; what matters is where the damage shows up.

Late-layer features act on the output directly, and we watch the damage grow with depth: Spearman $\rho = 0.97$ on Gemma-2-2B BatchTopK, $0.70$ for GemmaScope on the same model, $0.81$ at 9B. Early-layer features are anchors — ablating them barely moves the local readout but disrupts layers above, and the anchoring effect runs the other way with depth ($\rho = -0.65$).

Two roles, one mechanism, split by depth — the control row in the opening figure sits flat near zero — at every depth, in every configuration.

The same token, two verdicts

The comparison at the top of this page is the whole result in one token. Here is what sits behind it.

Deleting the GemmaScope feature costs 0.38 in logit. The BatchTopK feature for the same token costs 2.72 — and nothing about the model changed between those two numbers. Across all 474 matched pairs the split is systematic, and the ordering is inconvenient — code and math tokens, the ones a tokenizer already treats as units, are the ones that survive the switch. Function words, where most steering and circuit work actually operates, agree least: 29 percent against 93.

GemmaScope and BatchTopK anchor 92 to 100 percent of their source layers — and their features recover 62 to 71 percent of the time. Both LlamaScope configurations invert it — they anchor 31 and 34 percent of layers, and recovery climbs to 97.7 and 95.5 percent: same tokens, matched by construction. One family’s features are load-bearing, the other’s are replaceable.

The prevalence gap points the same way — 46 times more single-token features under GemmaScope than LlamaScope (at comparable 8 to 9B scale). We treat that number as a magnitude bound rather than an attribution — because the two sides differ in base model, tokenizer, training corpus, dictionary width, and a post-hoc activation conversion all at once.

It is not the activation function

The obvious explanation is the activation function (Rajamanoharan et al., 2024): GemmaScope is JumpReLU (Lieberum et al., 2024; Rajamanoharan, Lieberum, et al., 2024), LlamaScope is TopK (Gao et al., 2025), and TopK’s competition among features (Makhzani & Frey, 2014) is exactly the kind of thing that would redistribute causal load.

It doesn’t hold up. Holding base model, layer, and dictionary width fixed and varying only the activation function ($N = 142$), the sign reverses — JumpReLU comes out ahead of TopK at $p = 0.036$. Meanwhile the token-matched BatchTopK (Bussmann et al., 2024) against GemmaScope comparison on the same base model ($N = 627$) goes the other way, $p = 1.2 \times 10^{-18}$.

Two comparisons on the same model, opposite signs. Whatever drives the family split — the activation function alone isn’t it. And that leaves the training recipe as the candidate we can’t rule out.

Is the detector doing the work?

Here is the objection a reviewer raised — and it’s the right one. We select features by how well their decoder vector aligns with a token embedding — then measure what happens to that same token: subtracting a direction that points at ” Paris” should hurt ” Paris” whether or not the feature is a validated single-token feature.

So we built the control that tests it — non-single-token features matched on decoder-to-embedding cosine — the exact axis the objection turns on.

Single-token necessity survives at eight of nine layer-configurations — where exact matching is possible, the matched controls are causally inert (315 in-band controls against 590 single-token features — one-sided $p = 6.1 \times 10^{-20}$). And alignment does not predict damage within the control population at all (Spearman $-0.024$ over 2,950 control ablations).

Limits

Our cross-family comparison co-varies training data, dictionary width, training recipe, and the TopK-to-JumpReLU conversion, so we lean on the within-model comparisons and the 46 times figure is a bound.

Our activation-based detection rests on a single fixed operating point (Heap et al., 2025). Our conclusions hold across a doubling of the thresholds — and our causal experiments use the independent percentile-based detector, but it’s still one operating point.ent percentile-based detector, but it’s still one operating point.

Single-token features are the endpoint: that’s where ground truth exists. Whether the instability extends to compositional features (Engels et al., 2024) is the obvious next question. We don’t know. Our claim is only this: if causal roles already diverge in the simplest matched case, we doubt the harder cases are more stable.

What to do differently

Three consequences follow for anyone who steers, edits, or builds circuits on SAE features:

Re-run your ablation checks under the family you actually deploy: necessity measured under one SAE doesn’t transfer to another — even on the same base model. That sharpens rather than softens the case that SAE latents are not canonical units of analysis (Chanin et al., 2024; Leask et al., 2025).

Profile the checkpoint rather than trusting its label: anchoring and recovery statistics are cheap — they told us more than the family name or the activation function did.

Treat per-feature causal necessity as something to measure: two SAEs trained on the same model can differ in causal structure. Current benchmarks (Karvonen et al., 2025) don’t surface that — evaluation-side work is pushing toward functional rather than reconstruction-based criteria (Braun et al., 2024; Makelov et al., 2024).

A feature’s causal necessity is real, but it’s not portable: treat the SAE family as an experimental variable to check, not a detail to abstract away.