Directions found after training, beside the anchor¶

Placing a concept during training needs an extra training term, and it lowers the task score somewhat. This report asks whether that placement does better than searching for the concept afterwards, on the same model and with the same scoring.

Findings¶

• The trade-off (E1) — On every run but one, each standard fitter climbs past the anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. removalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better. before it reaches the spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. criterion, mostly through edits at the example answers. With a configuration chosen on the other seven runs instead, the fitters remove a little less, and on one run in eight a choice made elsewhere misses the slice where that run holds the op.
• Spill at matched removal (H1) — Misses. Turned up to the anchored removal, no standard fitter spills past the criterion on more than one run, so each is accepted.
• Removal at the spill criterion (H2) — Misses. LEACELEACE Least-squares concept erasure (Belrose et al., 2023): the smallest affine edit, measured in the covariance-whitened norm, after which no linear probe can read the target. For one target it works with two directions: c, the covariance of the states with the target (for a binary target, the difference in means), and Σ⁻¹c, where Σ is the covariance of the states, which is the direction a least-squares probe reads. It reads how much of the concept a state holds along Σ⁻¹c and moves the state back along c until that reading is the mean, so afterwards the probe reads the same value for every state. The two directions differ, so the edit is oblique (not a perpendicular projection). and INLP remove more than the anchor within the criterion, the probe is Unresolved, and difference in meansDifference in means The unit vector from the mean state of the other contexts to the mean state of difference contexts. The direction most steering work uses. The states are unit vectors, so their mean lies inside the sphere, but only the direction of the difference is used, and the edit puts each state back on the sphere. is left out under H4; with the pooled fits alone, LEACE is close to the anchor and the others remove less.
• The ceiling against the fitters (H3) — Misses. The optimized directionOptimized direction A unit vector per edited slice, fitted by gradient descent on the control so that the edited answers on difference contexts move toward the target null while the answers on the other contexts stay where they were. It uses the target null, which no real pipeline has, so it is a diagnostic ceiling: about the best a rank-1 edit (an edit along one direction) of this control could do with the scoring in hand. INLP edits a span of several directions, so this ceiling does not bound it. COAST is its nearest published relative., which acts at every position, removes less than the rank-1 fitters that edit the example answers.
• The ceiling against the anchor (S1) — Matches. Net of the axis-median baseline that favors it, the optimized direction removes about as much as the anchor, and its two runs over the spill criterion are within chance of the anchored none.
• The fitters find a placed concept (H4) — Partial. The probe, LEACE, and INLP find the placed concept; difference in means misses the gate by less than 0.001.
• Grading, landing, and where the answers go (E2) — The edits grade alike and send the answers to the same places; the fitters land a little further toward the target nullTarget null Where a removal should land: what an ideal predictor that had lost one op and nothing else would answer. It is the Bayes posterior over ops given the examples, with the removed op set to zero and the rest renormalized, so it still weighs the other ops by how well they fit. than the anchor.
• How many labels a fitter needs (E3) — Difference in means, LEACE, and INLP keep their removal down to a fiftieth of the labels, and the probe drops to 0.84.
• The write against each bound (E4) — The writes stay within their bounds apart from three sites on two runs, though most post-hoc edits act at one slice, where the write equals the bound.
• What a fresh probe can still find (E5) — No edit brings a fresh probe near chance, LEACE included.
• The two baseline rules (E6) — The two baselines disagree at configurations far past the spill criterion; at the points the verdicts read, no verdict of H1 or H2 changes under either one alone, and S1 would fall short under the permuted-label rule alone.
• Without role labels — Restricted to fits that need no role labels, LEACE and INLP remove about as much as the anchor and spill past the criterion at matched removal on two runs in eight against none; difference in means removes less and spills on six. Drawn again with these fits, E1 to E5 show the same picture, with the fitters a little lower.

Why this experiment¶

The anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. pipeline trains the model with a pull toward the first basis vector e₁ on labelled difference contexts, and removes the op afterwards by projecting e₁ out of the residual stream¹1 Residual stream: the running vector at each position, which every block reads from and adds its output to. at every slice and position. On the recipe of record, the fixed-effectsfixed-effects The full scoring suite on the recipe of record at seeds 712 to 719: removal 0.88 at the full dose, spill below the criterion at every dose, graded drops, a landing of 0.50, and a write within the bound. experiment found that this edit removes most of the op and costs the other ops less than the selectivity criterion allows. It also drops each context by about as much as that context needed difference, and the blocks after the edit keep what it writes within a bound set before the edit runs.

A reader who knows the steering and erasure literature will ask whether a direction found after training does as well. A conventional pipeline trains with no pull and fits a direction for the concept post hoc, from the states and the labels. It then projects that direction out, replaces the component along it with a mean, or erases it with LEACELEACE Least-squares concept erasure (Belrose et al., 2023): the smallest affine edit, measured in the covariance-whitened norm, after which no linear probe can read the target. For one target it works with two directions: c, the covariance of the states with the target (for a binary target, the difference in means), and Σ⁻¹c, where Σ is the covariance of the states, which is the direction a least-squares probe reads. It reads how much of the concept a state holds along Σ⁻¹c and moves the state back along c until that reading is the mean, so afterwards the probe reads the same value for every state. The two directions differ, so the edit is oblique (not a perpendicular projection).. If one of those matched the performance of the anchor on this model, we would have to say that the placement added nothing a search could not find, in a model of this class.

Ex-2.2.1ex-2.2.1 On the D2.1 checkpoints, every fitted direction that removed at least half of red from the control cost the lines with no red 0.37 or more, an order of magnitude above the anchored edit. asked this once, for red on the D2.1 checkpoints. Every fitted direction there that removed at least half of red cost the lines with no red far more than the anchored edit did. But that run gave the fitters little: three control seeds, labels from the token alone, one fit per slice, and no tuning. So this report gives the fitters every advantage we can, scores them the same way as the anchor, and asks where they fall on the trade-off between removalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better. and spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02..

Runs and methods¶

No new run is trained. The anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. runs are from the recipe of record: the eight τ_ℓ fixed runs of the fixed confirmationfixed confirmation τ_ℓ fixed at fresh seeds 712 to 719 keeps every anchor; it is the recipe of record, and its checkpoints and dose scoring are stored. at seeds 712 to 719. The controls are the eight runs of the anneal confirmationanneal confirmation Trained the controls at seeds 712 to 719 that this comparison fits on, sharing initialization, batches, and corpus with the anchored runs. at the same seeds, trained with no pull, with the same initialization, batches, and corpus. Every post-hoc method is fitted on a control and scored on it, so each method has eight runs, and each comparison with the anchor is between two samples of eight runs.

Anchored

The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment.

Anchored, fixed labels

The same recipe trained on one 1/50 label draw held for the whole run, at the same seeds (the fixed-labels experiment), under the same edit. It is a robustness row beside the main table, from its stored scores; the hypotheses judge the anchored arm above, whose labels are drawn again at each visit, as one pass over a large corpus would draw them.

Difference in means

The unit vector from the mean state of the other contexts to the mean state of difference contexts. The direction most steering work uses. The states are unit vectors, so their mean lies inside the sphere, but only the direction of the difference is used, and the edit puts each state back on the sphere.

Ridge probe

The normalized weight vector of a linear readout of the target, fitted by least squares with an ℓ₂ penalty. It finds what can be read off the state linearly.

LEACE

Least-squares concept erasure (Belrose et al., 2023): the smallest affine edit, measured in the covariance-whitened norm, after which no linear probe can read the target. For one target it works with two directions: c, the covariance of the states with the target (for a binary target, the difference in means), and Σ⁻¹c, where Σ is the covariance of the states, which is the direction a least-squares probe reads. It reads how much of the concept a state holds along Σ⁻¹c and moves the state back along c until that reading is the mean, so afterwards the probe reads the same value for every state. The two directions differ, so the edit is oblique (not a perpendicular projection).

Iterated probe (INLP)

The ridge probe fitted, projected out, and fitted again on what is left, to a rank of 2, 4, or 8; the edit removes the whole span. It covers the case of a concept held across several directions, which an edit along one could not remove.

Optimized direction

A unit vector per edited slice, fitted by gradient descent on the control so that the edited answers on difference contexts move toward the target null while the answers on the other contexts stay where they were. It uses the target null, which no real pipeline has, so it is a diagnostic ceiling: about the best a rank-1 edit (an edit along one direction) of this control could do with the scoring in hand. INLP edits a span of several directions, so this ceiling does not bound it. COAST is its nearest published relative.

The first four are the standard fitters that H1 and H2 judge. Each is also fitted on the anchored runs, where training placed the concept on e₁, to check that it can find a concept that is there: it must remove about as much as the edit along e₁ does there, within the same spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. limit. That check is H4, and a fitter that does not get through it is left out of the verdicts of H1, H2, H3, and S1. The sans-diff modelsans-diff model A model trained with no loss on difference comes 0.58 of the way from the control to the target null, against 0.50 for the edited recipe of record, and misses in the same kind of way., trained with no loss on difference, stays the reference for how close to the target nullTarget null Where a removal should land: what an ideal predictor that had lost one op and nothing else would answer. It is the Bayes posterior over ops given the examples, with the removed op set to zero and the rest renormalized, so it still weighs the other ops by how well they fit. a model of this size comes.

What each method gets¶

We use three sets of contexts, and no context is in more than one: a fit setFit set 280,000 fresh contexts, 40,000 per op, drawn from the training generator of the run with its replacement op noise: about the size of the training corpus, and drawn the same way. Each comes with its generating op and its posterior over ops. The states are taken from the clean pass of the run at every slice and position. The fits are made from means and covariances accumulated over the set, so the states need not be held in memory at once. the directions are fitted on, a tuning setTuning set Another 14,000 fresh contexts from the same generator, drawn the same way. the configuration is chosen on, and the test setTest set The 14,000 held-out contexts of ex-2.2.21, 2,000 per op, so the anchored numbers need no recomputing. every published D2.2 score is taken on.

Fit set

280,000 fresh contexts, 40,000 per op, drawn from the training generator of the run with its replacement op noise: about the size of the training corpus, and drawn the same way. Each comes with its generating op and its posterior over ops. The states are taken from the clean pass of the run at every slice and position. The fits are made from means and covariances accumulated over the set, so the states need not be held in memory at once.

Tuning set

Another 14,000 fresh contexts from the same generator, drawn the same way.

Test set

The 14,000 held-out contexts of ex-2.2.21, 2,000 per op, so the anchored numbers need no recomputing.

Every method is fitted with a label on every difference context. The anchor had a label on a fiftieth of them at each visit, but the labels were drawn again at every visit. So over 400 epochs about three quarters of the difference contexts were labelled at some point, about one and a half times each.

So over the whole run the anchor met about as many distinct labelled contexts as the fit set holds, but parity in that count is no tie. The anchor pull saw one difference context in fifty on each pass over the corpus, a few hundred at a time, through a stochastic term, while the fitters see every label at once, in closed form. What the fitters lack is a term that shaped the whole representation around the concept, and that is what this experiment measures. The label ladder of E3 measures how much the label count itself matters to them.

The fitters also get a graded target: the posterior on difference given the examples. A binary label marks the generating op even where the examples point to another op, and the graded target takes away that noise.

Each method then gets a grid over how it is fitted, where it acts, and how much it removes:

The fits at the example answers, the query =, and the query operands use role labels a real corpus does not have. So they are generous diagnostics rather than candidate recipes, and the tables mark which sites each selected configuration used.

The anchor gets no grid. Its one configuration is the published edit, which was reached through some thirty experiments at other seeds. The fitters arrive with their own literatures behind them, and get a per-run grid on top.

Selection rule (frozen)

Per method and per run, every configuration is scored on the tuning set, net of the median drop under the same operator form along fifteen fixed coordinate axes other than e₁, and the frontier is taken: the configurations that no other beats on both removal and spill. Two points are read off the frontier. Removal at the spill criterion is where the frontier crosses a spill of 0.02, interpolated linearly between the two frontier configurations that bracket it; if no configuration is within the criterion it is the one with the least spill, and if every one is within it it is the one with the most removal. Spill at matched removal is where the frontier crosses the seed mean of the anchored removal at full dose, interpolated the same way; a frontier that never reaches that removal has no such point, and the run is recorded as not matched. The bracketing configurations are then scored on the test set, and each point is their test scores mixed with the weights found on the tuning set, so the test set chooses nothing, and the point can sit a little off the line on the test set; the table shows where it sits.

RemovalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better. at the spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. criterion mirrors what a practitioner with one of these methods would do: sweep the layer and the strength on a validation set, keep the side effects within a tolerance, and take the strongest edit that does.

Measurements¶

Every measurement is defined as in the D2.2 design and fixed-effectsfixed-effects The full scoring suite on the recipe of record at seeds 712 to 719: removal 0.88 at the full dose, spill below the criterion at every dose, graded drops, a landing of 0.50, and a write within the bound., and taken through sca.intervention.apply. So any method that produces a subspace and an operator is scored by the code that scores the anchor.

Removal

The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better.

Spill

The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02.

Net and raw

A net drop has a baseline drop subtracted; a raw drop does not. Published anchored scores are net of the median drop under the same edit along each of the other 63 coordinate axes of the run. A fitted direction is not an axis, so each post-hoc configuration gets a baseline in its own terms: the median drop over fifteen fits of the same fitter with the labels permuted across contexts. A permuted fit gives a direction with no concept in it, and the same kind of operator. On the controls the coordinate-axis median is computed too, and each hypothesis is judged on whichever of the two baselines favors the post-hoc method for that measure (the smaller for removal, the larger for spill). The anchor keeps its published baseline. Raw drops are shown beside net ones throughout.

Task change

The change in task score from the mean of the controls. It is zero for every post-hoc method, since the model is the control. For the anchor it is the published change, and the table of E1 shows it as the price of anchoring.

The trade-off (E1)¶

Each hypothesis takes a point from this trade-off, so it comes first. The figure draws the frontier of each run on the test setTest set The 14,000 held-out contexts of ex-2.2.21, 2,000 per op, so the anchored numbers need no recomputing., with the point the selection rule read on it, beside the anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. dose curve of each run. The table after it gives, per method, the two points of the selection rule, the task changeTask change The change in task score from the mean of the controls. It is zero for every post-hoc method, since the model is the control. For the anchor it is the published change, and the table of E1 shows it as the price of anchoring., and how many runs chose each site.

Each mark is net of the median drop along the fifteen coordinate axes of the tuning setTuning set Another 14,000 fresh contexts from the same generator, drawn the same way. under the same operator form, the baseline the selection rule uses on the tuning set. We use it here because the two test-set baselines were computed only at the configurations the rule chose.

On every run but one, each standard fitter has a frontier that climbs past the anchored removalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better. before it reaches the spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. criterion. The configuration it picks at the criterion removes nearly all of difference.

On every run, the configuration each fitter chose at the criterion was fitted at the example answers and edits there alone; the table below counts the sites. The optimized directionOptimized direction A unit vector per edited slice, fitted by gradient descent on the control so that the edited answers on difference contexts move toward the target null while the answers on the other contexts stay where they were. It uses the target null, which no real pipeline has, so it is a diagnostic ceiling: about the best a rank-1 edit (an edit along one direction) of this control could do with the scoring in hand. INLP edits a span of several directions, so this ceiling does not bound it. COAST is its nearest published relative., which acts at every position by design, stops about where the anchor does.

So on this model a direction fitted after training, with every label and a per-run grid, removes more of difference within the criterion than the anchored edit does, and spills no more at matched removal. It costs the task nothing, since the model is the control, while the anchored runs score 0.003 lower than the controls. The fixed-labelsfixed-labels Holding one label draw fixed for the run, as a real labelled set would be, keeps the anchor at every rate tried, but a few runs end with the edit removing much less of the op. row removes less than the main anchored arm, as that experiment found.

The edge of the fitters comes mostly from fitting and editing at the example answers, a role a real corpus does not label. Restricted to the pooled fits, which need no role labels, the fitters lose most of that edge (Without role labels).

One configuration chosen on other runs¶

Each run above picks its own configuration, among about a thousand, on the model it edits. The anchor had no such choice on these runs: its recipe was settled on earlier seeds and run on these eight as it was. The next figure puts the fitters on the same terms. For each run in turn, the selection rule picks a configuration on the other seven, off the frontier of their seed means on the tuning set, and the fitter is then fitted on the run left out with that configuration. So the choice is made eight times per method: seed 712 takes the configuration chosen on 713 to 719, seed 713 the one chosen on 712 and 714 to 719, and so on. This analysis was not preregistered.

With the configuration chosen on other runs, the fitters remove about 0.86 to 0.94 on seven of the eight runs, a little less than with a choice of their own (0.93 to 0.96), and land past the spill criterion on three or four of the eight. A point picked right at the criterion on other runs lands on either side of it about as often, while the anchor sits well inside it on every run.

The eighth run, seed 712, is where the choice fails to carry over. Difference in meansDifference in means The unit vector from the mean state of the other contexts to the mean state of difference contexts. The direction most steering work uses. The states are unit vectors, so their mean lies inside the sphere, but only the direction of the difference is used, and the edit puts each state back on the sphere., LEACELEACE Least-squares concept erasure (Belrose et al., 2023): the smallest affine edit, measured in the covariance-whitened norm, after which no linear probe can read the target. For one target it works with two directions: c, the covariance of the states with the target (for a binary target, the difference in means), and Σ⁻¹c, where Σ is the covariance of the states, which is the direction a least-squares probe reads. It reads how much of the concept a state holds along Σ⁻¹c and moves the state back along c until that reading is the mean, so afterwards the probe reads the same value for every state. The two directions differ, so the edit is oblique (not a perpendicular projection)., and INLP keep almost nothing of their removal there (0.00 to 0.03): their choices on the other runs edit slice 2, and on this run their own choices edit slice 1. The probe keeps 0.97 there, with an edit over slices 1 to 3. So where a post-hoc edit works moves from run to run, by a slice on one run in eight here, and a fixed recipe for a fitter has to cover that, while the anchor puts the concept on e₁ on every run. One run is a small sample; a larger set of seeds would say how often this happens.

A practitioner would choose with some room under the criterion rather than at it. With the choice made at a spill of 0.01 on the other runs, the fitters land past the criterion on at most three of the eight runs (LEACE on none), and remove about as much as before on the seven. So most of the runs over the criterion come from choosing at the line. The miss on seed 712 stays: the choice made on the other runs still edits a slice where that run does not hold the op. This check was added after the run, after a second reader suggested it.

Spill at matched removal (H1)¶

What we expect. If we turn each fitter up until it removes as much of difference as the anchor does, then on most runs it will cost another op more than the criterion allows.

The rule. For each fitter we count the runs, out of eight, whose spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. at matched removalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better. is over 0.02 net. A run whose frontier never reaches the anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. removal counts as over, since that fitter cannot remove as much at any spill. For the anchor we count the runs whose spill at full dose is over 0.02 net. A one-sided Fisher exact test²2 Fisher exact test: a test of whether two counts out of the same number of trials differ by more than chance. With the anchor at none of eight, a fitter at four or more of eight gives p below 0.05. then says whether the fitter count is higher than the anchored count by more than chance would give. A fitter is rejected if its count is five or more and the test gives p below 0.05, and accepted if its count is under five. A count of five or more that the test cannot tell apart from the anchored count (which can happen only if the anchor itself spills on one or more runs; five against one gives p of 0.06) would be outside the plan, and that fitter would be Unresolved. Among the fitters accepted under H4, H1 passes if every fitter with a resolved verdict was rejected, misses if every one was accepted, and is partial otherwise; it is Unresolved if no fitter is accepted under H4 or none has a resolved verdict. The Findings line names any fitter left out or Unresolved. A fitter accepted here would mean that, on this model, a searched direction serves as well as a placed one.

The figure shows spill at matched removal for each run, with the anchor at full dose beside the fitters.

Misses. No standard fitter spills past the criterion on more than one run once it is turned up to the anchored removal, so each is accepted, and H1 misses. Difference in meansDifference in means The unit vector from the mean state of the other contexts to the mean state of difference contexts. The direction most steering work uses. The states are unit vectors, so their mean lies inside the sphere, but only the direction of the difference is used, and the edit puts each state back on the sphere. is left out of the verdict under H4, and it would have been accepted too. Turned up that far, a fitted direction costs the other ops about as little as the anchored edit does.

Removal at the spill criterion (H2)¶

What we expect. If we hold each fitter to the same spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. limit as the anchor, then it will remove less of difference than the anchor does.

The rule. For each fitter we take the difference of seed means between the anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. removalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better. (at full dose, or where its own dose curve crosses the criterion if it spills past it) and the removal at the spill criterion of the fitter, with a 95% Welch t interval³3 Welch t interval: a 95% confidence interval for the gap between two means that lets the two groups scatter by different amounts. An interval wholly above zero means the eight anchored runs remove more than the eight fitted ones by more than seed-to-seed scatter accounts for.. A fitter is rejected if the interval lies wholly above zero (the anchor removes more). It is accepted if the interval lies wholly below zero (the fitter removes more), or contains zero and lies within 0.05 of it on both sides (the two remove about as much). An interval that contains zero and reaches further than 0.05 on either side would mean the seeds could not tell, and that fitter would be Unresolved. Among the fitters accepted under H4, H2 passes if every fitter with a resolved verdict was rejected, misses if every one was accepted, and is partial otherwise; it is Unresolved if no fitter is accepted under H4 or none has a resolved verdict. The Findings line names any fitter left out or Unresolved.

A small edge for a fitter would get it accepted under this rule, and it would not surprise us. Every run writes the evidence for the op partly off e₁ (the low-runslow-runs In every run the evidence that an example fits difference lies partly on e₁ and partly off it, and the edit along e₁ never touches the off-axis part. experiment). So the edit along e₁ leaves a little behind, while a fitted direction can take in both parts.

The figure shows removal at the spill criterion for each run.

Misses. LEACELEACE Least-squares concept erasure (Belrose et al., 2023): the smallest affine edit, measured in the covariance-whitened norm, after which no linear probe can read the target. For one target it works with two directions: c, the covariance of the states with the target (for a binary target, the difference in means), and Σ⁻¹c, where Σ is the covariance of the states, which is the direction a least-squares probe reads. It reads how much of the concept a state holds along Σ⁻¹c and moves the state back along c until that reading is the mean, so afterwards the probe reads the same value for every state. The two directions differ, so the edit is oblique (not a perpendicular projection). and INLP remove more than the anchor within the criterion, by more than the seeds scatter, so both are accepted. The probe removes more on seven runs but stops far short on one, which widens its interval to span −0.12 to just above zero. That interval contains zero but reaches further than 0.05 from it, so the verdict of the probe is Unresolved.

Every resolved verdict is an acceptance, so H2 misses. Difference in meansDifference in means The unit vector from the mean state of the other contexts to the mean state of difference contexts. The direction most steering work uses. The states are unit vectors, so their mean lies inside the sphere, but only the direction of the difference is used, and the edit puts each state back on the sphere. would have been accepted too, but it is left out under H4.

We expected a small edge for a fitter, since part of the evidence lies off e₁. The edge here is larger, and it comes mostly from where the fitters act: the rule picks the configurations fitted at the example answers, which edit those positions alone and use role labels a real corpus does not have.

Without role labels repeats the comparison with every fitter restricted to the pooled fits, which is the like-for-like reading for the anchor.

The ceiling against the fitters (H3)¶

What we expect. If we search for a direction with the scoring in hand, then it does better than every standard rank-1 fitter.

The rule. The comparison takes the rank-1 fitters accepted under H4: difference in meansDifference in means The unit vector from the mean state of the other contexts to the mean state of difference contexts. The direction most steering work uses. The states are unit vectors, so their mean lies inside the sphere, but only the direction of the difference is used, and the edit puts each state back on the sphere., the probe, and LEACELEACE Least-squares concept erasure (Belrose et al., 2023): the smallest affine edit, measured in the covariance-whitened norm, after which no linear probe can read the target. For one target it works with two directions: c, the covariance of the states with the target (for a binary target, the difference in means), and Σ⁻¹c, where Σ is the covariance of the states, which is the direction a least-squares probe reads. It reads how much of the concept a state holds along Σ⁻¹c and moves the state back along c until that reading is the mean, so afterwards the probe reads the same value for every state. The two directions differ, so the edit is oblique (not a perpendicular projection).. RemovalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better. by the optimized directionOptimized direction A unit vector per edited slice, fitted by gradient descent on the control so that the edited answers on difference contexts move toward the target null while the answers on the other contexts stay where they were. It uses the target null, which no real pipeline has, so it is a diagnostic ceiling: about the best a rank-1 edit (an edit along one direction) of this control could do with the scoring in hand. INLP edits a span of several directions, so this ceiling does not bound it. COAST is its nearest published relative. at the spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. criterion is at least the highest seed mean among them, and its count over the criterion at matched removal (as in H1) is at most the lowest. It passes if both hold, and misses if neither does; one without the other would be outside the plan, and the verdict would be Unresolved, as it would be if no rank-1 fitter is accepted under H4. The comparison is between seed means and counts, with no interval, which is enough for a check that the ceiling is a ceiling.

INLP is left out of H3 because it edits a span of two to eight directions, which a rank-1 ceiling does not bound; its two measures are shown beside the others. If INLP does better than the ceiling, the control holds the op across several directions, which E5 looks at from another side.

LEACE is rank-1 but oblique, and the ceiling is a perpendicular edit, so LEACE could in principle beat it too. We keep it in, since LEACE beating the ceiling would say the oblique form matters.

Misses. The optimized direction removes less than any of the rank-1 fitters it is compared with, and is over the criterion at matched removal on more runs. Both shortfalls come from two runs on which its frontier never reaches the anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. removal. INLP, outside the rule, does better than it too.

So it is no ceiling for the fitters here, for the reason seen in H2: the search acts at every position, as it was planned to, while the fitters picked edits at the example answers alone. Against the pooled fits (Without role labels) it does about as well as the best of them.

The ceiling against the anchor (S1)¶

What we expect. This section predicts nothing: it is a frozen rule that decides between two outcomes, and its badge is Decided, or Unresolved. Either the best rank-1 search on the control does as well as the anchor, or it still falls short.

The rule. The ceiling matches the anchor if the 95% Welch interval³ on the difference of seed means in removalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better. at the spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. criterion contains zero and lies within 0.05 of it on both sides (as for an accepted fitter in H2), and the Fisher test² cannot tell its count over the criterion at matched removal apart from the anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. count; then the advantage of placing the concept is that no search is needed. The ceiling falls short if the interval lies wholly above zero; then the control holds the op in a way no rank-1 edit removes cleanly, which is the stronger claim for the anchor. Any other result (the interval wholly below zero, an interval that contains zero but reaches further than 0.05, or a count the Fisher test does tell apart) would be outside the plan, and the verdict would be Unresolved.

Matches. The interval on the gap in removal straddles zero and lies within 0.05 of it, and the Fisher test cannot tell the counts apart, so the best rank-1 search acting at every position does about as well as the anchor. By the frozen rule, the advantage of placing the concept is then that no search is needed.

That conclusion takes the search as the reference. H2 shows that fitters acting at the example answers do better still, and we come back to this in the discussion.

The removal here is net of the axis median, the baseline that favors the method. Under the permuted-label baseline alone the optimized directionOptimized direction A unit vector per edited slice, fitted by gradient descent on the control so that the edited answers on difference contexts move toward the target null while the answers on the other contexts stay where they were. It uses the target null, which no real pipeline has, so it is a diagnostic ceiling: about the best a rank-1 edit (an edit along one direction) of this control could do with the scoring in hand. INLP edits a span of several directions, so this ceiling does not bound it. COAST is its nearest published relative. would remove far less, and the ceiling would fall short (E6).

The fitters find a placed concept (H4)¶

What we expect. If we fit on the anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. runs, where training placed the op on e₁, then each standard fitter removes about as much as the edit along e₁ does, within the same spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. limit. A fitter that cannot do that where the concept is known to be is no fair stand-in for its method on the controls.

The rule. Restricted to fits on the pooled positions or at the example answers, a fitter is accepted if its removalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better. at the spill criterion on the anchored runs is no more than 0.1 below what the edit along e₁ removes at full dose on the same runs (seed means), and rejected otherwise. A fitter that finds e₁ but has to stop short of it to stay within the criterion, as INLP may when its extra directions cost spill, is rejected. H4 passes if every standard fitter is accepted, misses if every one is rejected, and is partial otherwise. On the anchored runs the fitter takes its permuted-label baseline, and the edit along e₁ keeps its published one. The query = is left out: the clean alignmentAlignment How far a state points along e₁: the cosine between the state and e₁, which for a unit-length state is its component along e₁; 1 on the anchor direction and 0 off it. there is near zero in the anchored runs (the mid-block-editmid-block-edit Editing the mid-block states as well gains nothing and spills; in passing it shows the query = near zero alignment throughout. experiment), so a fit at that site is not expected to find e₁.

Beside the verdict, and not part of it, we report how close each fitted direction is to e₁: the cosine with e₁⁴4 Cosine with e₁: how closely a direction lines up with e₁, 1 when it lies along e₁ and 0 when it is perpendicular to it. at each of slices 1 to 3, for the target and site of the selected configuration, given as the seed mean of its absolute value. For INLP it is the largest cosine between e₁ and the span.

The rule for a rejected fitter, frozen here: a fitter rejected under H4 is left out of the verdicts of H1, H2, H3, and S1, its rows stay in the tables of every section, greyed out, and the report names it as a method the comparison could not fairly score. A fitter that cannot remove a placed concept would likely do badly on the controls too, so leaving it out leans toward the fitters, which is the cautious direction for a claim about the anchor.

The figure shows the removal at the spill criterion of each fitter on the anchored runs, restricted to the two sites, beside the edit along e₁.

Partial. The probe, LEACELEACE Least-squares concept erasure (Belrose et al., 2023): the smallest affine edit, measured in the covariance-whitened norm, after which no linear probe can read the target. For one target it works with two directions: c, the covariance of the states with the target (for a binary target, the difference in means), and Σ⁻¹c, where Σ is the covariance of the states, which is the direction a least-squares probe reads. It reads how much of the concept a state holds along Σ⁻¹c and moves the state back along c until that reading is the mean, so afterwards the probe reads the same value for every state. The two directions differ, so the edit is oblique (not a perpendicular projection)., and INLP each remove within 0.1 of what the edit along e₁ removes, and LEACE more. Difference in meansDifference in means The unit vector from the mean state of the other contexts to the mean state of difference contexts. The direction most steering work uses. The states are unit vectors, so their mean lies inside the sphere, but only the direction of the difference is used, and the edit puts each state back on the sphere. falls short of the gate by less than 0.001 (0.782 against 0.783; both print as 0.78 in the table), so it is rejected and left out of the other verdicts.

That rejection is fragile, and its cause is the baseline more than the fitter. Raw, difference in means removes as much as the others; it is the permuted-label baseline that takes it under the gate. Net of the coordinate-axis median instead, the baseline the anchored edit itself is scored against, difference in means removes 0.97, well above the gate, and every fitter would be accepted (removals 0.87 to 0.97). Had it been kept in, H1 and H2 would have accepted it too, so leaving it out changes no verdict. This reading under the axis baseline was added after the run.

On the anchored runs a permuted fit is not neutral. The difference of the means of two random halves of the states points along the directions where the states vary most, and training made e₁ one of those. So a permuted fit removes part of the op too, which puts the net removal of every fitter on these runs below its raw one. E6 compares the baselines on the controls.

The directions of difference in means, LEACE, and INLP on these runs point well toward e₁ at the slice where the fit was made, without lying on it, as the low-runslow-runs In every run the evidence that an example fits difference lies partly on e₁ and partly off it, and the edit along e₁ never touches the off-axis part. experiment led us to expect. The probe points much less toward e₁ and still removes the op, likely because a probe turns its direction away from where the states vary most, and training made e₁ one of those directions.

Grading, landing, and where the answers go (E2)¶

At the configuration each method selects for removalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better. at the spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. criterion, we run the scoring suite of the fixed-effectsfixed-effects The full scoring suite on the recipe of record at seeds 712 to 719: removal 0.88 at the full dose, spill below the criterion at every dose, graded drops, a landing of 0.50, and a write within the bound. experiment. The anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. edit is scored on the same held-out contexts as the other methods.

These measures read each context on its own, where H1 to H4 average the drop over all the contexts of an op. The baselines were taken per op, so they cannot be subtracted context by context, and the drops here are raw.

Grading is the share of the way to the target nullTarget null Where a removal should land: what an ideal predictor that had lost one op and nothing else would answer. It is the Bayes posterior over ops given the examples, with the removed op set to zero and the rest renormalized, so it still weighs the other ops by how well they fit. removed in each bin of the posterior on difference. Landing is how far the edit moves the answers toward the target null, in total variationTotal variation A distance between two answer distributions: half the summed absolute difference, from 0 (identical) to 1 (no overlap). Mass a model puts on tokens that are not colors counts as distance.. A matrix of where the edited model puts its answers on difference contexts shows where they go.

Every edit grades its drop by how sure the context is of difference, and they all grade about alike. The share of the way to the target null removed stays between 0.82 and 0.98 in every bin, so each edit drops a context by about as much as that context needed difference. The share is lowest where the context is least sure, 0.82 to 0.88 below a posterior of 0.2, and highest between 0.5 and 0.8, at 0.96 to 0.98.

It dips a little for the surest contexts, most for the probe and the optimized directionOptimized direction A unit vector per edited slice, fitted by gradient descent on the control so that the edited answers on difference contexts move toward the target null while the answers on the other contexts stay where they were. It uses the target null, which no real pipeline has, so it is a diagnostic ceiling: about the best a rank-1 edit (an edit along one direction) of this control could do with the scoring in hand. INLP edits a span of several directions, so this ceiling does not bound it. COAST is its nearest published relative.. The anchor and INLP sit at the top in every bin, and the optimized direction at the bottom.

The fitters land a little further toward the target null than the anchor: 0.54 to 0.55 of the clean distance, against 0.50. The optimized direction, which is fitted toward the target null, lands furthest at 0.58, about where the sans-diff modelsans-diff model A model trained with no loss on difference comes 0.58 of the way from the control to the target null, against 0.50 for the edited recipe of record, and misses in the same kind of way. lands. The ranges over runs overlap.

None of the edits gets all the way, and they fall short in the same way. The answers they move off difference go to the answers of the other ops in about the proportions the target null gives, a fifth to a quarter each, as do those of the sans-diff model, which was trained with no loss on difference; and 0.03 to 0.06 stays on the answers of difference itself.

The anchor puts a little less than the methods or the target null on the answers of lighten, and a little more on those of sat-hsv. On grading and on where the answers go, the anchored edit and the post-hoc edits are hard to tell apart.

How many labels a fitter needs (E3)¶

At the configuration each method selects, we repeat the fit with labels on every difference context of the fit setFit set 280,000 fresh contexts, 40,000 per op, drawn from the training generator of the run with its replacement op noise: about the size of the training corpus, and drawn the same way. Each comes with its generating op and its posterior over ops. The states are taken from the clean pass of the run at every slice and position. The fits are made from means and covariances accumulated over the set, so the states need not be held in memory at once., then on a fixed half, a tenth, a twenty-fifth, and a fiftieth of them. The last two pair with the anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. runs of the fixed-labelsfixed-labels Holding one label draw fixed for the run, as a real labelled set would be, keeps the anchor at every rate tried, but a few runs end with the edit removing much less of the op. experiment, which held one draw at those rates. The unlabelled difference contexts are counted among the other contexts, as an unlabelled context is for the anchor.

A fixed fiftieth is 800 contexts: about what the anchor met in one pass over the corpus, and far fewer than it met over training. So in that respect the lowest rate is harsher than what the anchor had. In another it is kinder: the fitters still see every label of the fixed set at once.

The removalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better. and spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. here are those of the configuration E2 to E5 read, with its own baselines: for each method, the lower-spill configuration of the pair that brackets removal at the spill criterion, which for every fitter is a fit at the example answers. So at every label count the numbers differ slightly from the points of H2, which interpolate between two configurations.

Difference in meansDifference in means The unit vector from the mean state of the other contexts to the mean state of difference contexts. The direction most steering work uses. The states are unit vectors, so their mean lies inside the sphere, but only the direction of the difference is used, and the edit puts each state back on the sphere., LEACELEACE Least-squares concept erasure (Belrose et al., 2023): the smallest affine edit, measured in the covariance-whitened norm, after which no linear probe can read the target. For one target it works with two directions: c, the covariance of the states with the target (for a binary target, the difference in means), and Σ⁻¹c, where Σ is the covariance of the states, which is the direction a least-squares probe reads. It reads how much of the concept a state holds along Σ⁻¹c and moves the state back along c until that reading is the mean, so afterwards the probe reads the same value for every state. The two directions differ, so the edit is oblique (not a perpendicular projection)., and INLP need few labels: from every label down to a fiftieth, their removal moves by less than 0.02, so a mean or a covariance over 800 labelled contexts already points the right way. The probe drops to 0.84 at a fiftieth, with one run at 0.49 and one over the spill criterion.

The anchor with fixed labels removes 0.81 at a fiftieth, below the three flat fitters at the same rate. So the fitters do not need many labels, and the number of labels is not where the anchor has an edge. The fitters at a fiftieth are fitted at the example answers, so the rate is matched with the anchor and the site is not.

The write against each bound (E4)¶

The write bound of the anchor does not apply as it stands to a direction that is not an axis, or to an oblique eraser, so each method gets the bound in its own terms: the angle its operator turns the clean states of the other ops through (the 99% quantile, per slice and role).

The write is the angle the operator turns those states through as they arrive, already changed by the edits at the earlier slices. Where the blocks after an edit put the concept back, the method shows a write over its own bound. A site counts as over when the write passes the bound by more than both tolerances of the D2.2 design: a quarter over the bound, and about a degree.

The write stays within its bound almost everywhere, though for most of the post-hoc edits there is little to measure. An edit at one slice has no edits before it, so its write equals its bound, and most of the configurations read here edit slice 2 alone.

Of the six runs where a method edits several slices, two show a site over: the probe at seed 716 and the optimized directionOptimized direction A unit vector per edited slice, fitted by gradient descent on the control so that the edited answers on difference contexts move toward the target null while the answers on the other contexts stay where they were. It uses the target null, which no real pipeline has, so it is a diagnostic ceiling: about the best a rank-1 edit (an edit along one direction) of this control could do with the scoring in hand. INLP edits a span of several directions, so this ceiling does not bound it. COAST is its nearest published relative. at seed 717. The anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. edit acts at every slice and shows no site over, as in the fixed-effectsfixed-effects The full scoring suite on the recipe of record at seeds 712 to 719: removal 0.88 at the full dose, spill below the criterion at every dose, graded drops, a landing of 0.50, and a write within the bound. experiment.

What a fresh probe can still find (E5)¶

LEACELEACE Least-squares concept erasure (Belrose et al., 2023): the smallest affine edit, measured in the covariance-whitened norm, after which no linear probe can read the target. For one target it works with two directions: c, the covariance of the states with the target (for a binary target, the difference in means), and Σ⁻¹c, where Σ is the covariance of the states, which is the direction a least-squares probe reads. It reads how much of the concept a state holds along Σ⁻¹c and moves the state back along c until that reading is the mean, so afterwards the probe reads the same value for every state. The two directions differ, so the edit is oblique (not a perpendicular projection). promises that no linear probe can read the concept after its edit, and the anchor promises nothing about that. Here we fit a fresh ridge probeRidge probe The normalized weight vector of a linear readout of the target, fitted by least squares with an ℓ₂ penalty. It finds what can be read off the state linearly. for the binary target on the edited states of the test setTest set The 14,000 held-out contexts of ex-2.2.21, 2,000 per op, so the anchored numbers need no recomputing., per slice and per method, with cross-validation over contexts.

The evidence for the op can be recomputed from the operand colors (the block1-editblock1-edit Block 1 writes the evidence for the op onto e₁ at the example answers, and a later block reads it at the query =. experiment), so a method may leave the op readable even where the answer no longer uses it. A probe that still finds the op does not mean the edit failed to remove it from the answer, and a probe that finds nothing does not bound what the edit writes.

We report it because it is the measure the erasure literature uses, and the one on which the anchor may come off worse. We expect LEACE to bring the probe near chance at the slices it edits, since that is what it is built to do.

We expect the anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. edit and the projections to leave the op readable well above chance, since the evidence can be recomputed and part of it lies off any one direction (the low-runslow-runs In every run the evidence that an example fits difference lies partly on e₁ and partly off it, and the edit along e₁ never touches the off-axis part. experiment). If the projections also bring the probe near chance, the op is held along fewer directions than that suggests.

The query = is the position whose output predicts the answer the task scores, so a model that uses the op must have it there in some form; the example answers are where block 1 writes the evidence for it (the block1-edit experiment). We score the probe by the area under the ROC curve⁵5 Area under the ROC curve (AUC): the chance that the probe scores a random difference context above a random context of another op. 0.5 is chance, 1 a perfect split. at the query = and at the example answers, pooled. It is fitted on half of 7,000 test contexts and scored on the other half.

No edit brings the probe near chance. Before any edit, a probe finds the op at the query = of slice 3 with an AUC of 0.90 on the control and 0.88 on the anchored model. After the edits it scores 0.64 to 0.71 there, lowest after the anchored edit.

The sans-diff modelsans-diff model A model trained with no loss on difference comes 0.58 of the way from the control to the target null, against 0.50 for the edited recipe of record, and misses in the same kind of way., trained with no loss on difference, leaves 0.67 for a fresh probe at the same site, about what the edits leave. So most of what the probe finds after an edit is there in a model that never learned the op, likely because the operand colors carry it.

At the example answers the order turns around: the anchored edit leaves the op more readable (0.79 at slice 2) than difference in meansDifference in means The unit vector from the mean state of the other contexts to the mean state of difference contexts. The direction most steering work uses. The states are unit vectors, so their mean lies inside the sphere, but only the direction of the difference is used, and the edit puts each state back on the sphere., LEACE, or INLP (0.70 to 0.74), which edit there.

LEACE does not do what we expected of it, and we do not know why. At the slice and site it edits, mostly at full dose, a fresh probe still reads 0.70, about what difference in means leaves.

Three candidates fit what we see: (a) The model keeps its states at unit length, so every edit ends by scaling the state back to length one. Rescaling by a length that depends on what was taken out can bring some of the op back along every other direction, while the guarantee of LEACE is for the linear edit alone. (b) Our LEACE takes a small ridge on a covariance that is close to rank-deficient (some directions barely vary), so it may leave low-variance directions that a fresh probe with a weak penalty can read. (c) On two runs it was fitted to the graded target, where the guarantee is for the posterior rather than the binary label the probe reads.

The two baseline rules (E6)¶

On the controls, we compare the permuted-label baseline with the coordinate-axis median, configuration by configuration, for the configurations scored on the test setTest set The 14,000 held-out contexts of ex-2.2.21, 2,000 per op, so the anchored numbers need no recomputing.. The Adam optimizer concentrates energy on some axes of a control, so the two can differ.

The baseline decides how much of a drop counts as the edit, and each hypothesis takes whichever rule favors the fitters. So if the two rules differ by about the size of a gap that a verdict turns on, that verdict rests on the choice of baseline.

The two rules disagree a lot on single configurations and little at the points the verdicts read. At some configurations a permuted-label fit costs every op about as much as the fitted edit does, while an edit along a coordinate axis costs almost nothing. Such a configuration shows little spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. under the permuted rule and a lot under the axis rule, well past the criterion: the marks far below the diagonal in the right panel.

At the points the verdicts read, the axis rule gives the higher removalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better. and the permuted rule the lower spill, so the favoring rule combines the two. Taken alone, the permuted rule lowers removal by 0.02 to 0.04 for the fitters and by 0.31 for the optimized directionOptimized direction A unit vector per edited slice, fitted by gradient descent on the control so that the edited answers on difference contexts move toward the target null while the answers on the other contexts stay where they were. It uses the target null, which no real pipeline has, so it is a diagnostic ceiling: about the best a rank-1 edit (an edit along one direction) of this control could do with the scoring in hand. INLP edits a span of several directions, so this ceiling does not bound it. COAST is its nearest published relative., whose permuted searches remove much of the op themselves. No verdict of H1 or H2 changes under either rule.

S1 is the exception. It uses the axis rule for removal, and under the permuted rule alone the optimized direction removes 0.58. The interval on the gap then lies wholly above zero (0.26 to 0.34), and the ceiling would fall short of the anchor.

A permuted search is not neutral either, since each context keeps its own target nullTarget null Where a removal should land: what an ideal predictor that had lost one op and nothing else would answer. It is the Bayes posterior over ops given the examples, with the removed op set to zero and the rest renormalized, so it still weighs the other ops by how well they fit. under the permutation. So the search still pulls the difference contexts that land in the permuted labelled set toward the null, against the keep term on the rest, and it finds a direction that removes much of the op.

Without role labels¶

The anchor uses no role labels. Its pull pools over the positions of a labelled context, and its edit acts at every position and slice. A fit at the example answers, the query =, or the query operands uses a role, which a real corpus does not label. So the like-for-like comparison restricts every fitter to the pooled fits: the selection rule runs again over the configurations fitted at every position, pooled, on the same runs and with the same baselines. The fitters keep a label on every difference context and a grid tuned per run over the target, slices, dose, refit, replacement, and rank, which the anchor does not have.

This reading was not preregistered, so it gives no verdict. The table applies the rules of H1 and H2 as a guide to how far apart the methods are. After it, the figures of E1 to E5 are drawn again with the pooled fits alone.

Without role labels the fitters lose most of their edge in removalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better.. LEACELEACE Least-squares concept erasure (Belrose et al., 2023): the smallest affine edit, measured in the covariance-whitened norm, after which no linear probe can read the target. For one target it works with two directions: c, the covariance of the states with the target (for a binary target, the difference in means), and Σ⁻¹c, where Σ is the covariance of the states, which is the direction a least-squares probe reads. It reads how much of the concept a state holds along Σ⁻¹c and moves the state back along c until that reading is the mean, so afterwards the probe reads the same value for every state. The two directions differ, so the edit is oblique (not a perpendicular projection). removes 0.91 against the anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. 0.88, with an interval on the gap of −0.07 to 0.01. INLP (0.89) and the probe (0.85) come close on average with a wider spread, and difference in meansDifference in means The unit vector from the mean state of the other contexts to the mean state of difference contexts. The direction most steering work uses. The states are unit vectors, so their mean lies inside the sphere, but only the direction of the difference is used, and the edit puts each state back on the sphere. removes 0.73, with one run at 0.17. Read by the rule of H2, every fitter's interval reaches further than 0.05 from zero on one side, so none is a match and none falls short.

At matched removal the order turns around. The anchored edit spillsSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. past the criterion on no run; LEACE and INLP do on two runs each, the probe on three, and difference in means on six, which the rule of H1 would reject. Every fitter reaches the anchored removal on every run, so these are runs that spill, not runs that fall short. The Fisher test cannot tell two runs in eight from none (p 0.23), so for LEACE and INLP this is a lean, not a separation.

So in the setting the anchor is built for, the best post-hoc eraser removes about as much as the anchored edit and, even with a grid tuned per run, spills past the criterion on a few more runs. Difference in means, the direction most steering work uses, removes less and spills more.

The trade-off with the pooled fits¶

As in E1, one frontier per run, now over the pooled fits alone, with the point the selection rule read on it. The optimized directionOptimized direction A unit vector per edited slice, fitted by gradient descent on the control so that the edited answers on difference contexts move toward the target null while the answers on the other contexts stay where they were. It uses the target null, which no real pipeline has, so it is a diagnostic ceiling: about the best a rank-1 edit (an edit along one direction) of this control could do with the scoring in hand. INLP edits a span of several directions, so this ceiling does not bound it. COAST is its nearest published relative. panel is the same as in E1.

The frontiers rise more slowly than at the example answers and level off lower, so each run settles for less removal at the criterion. A configuration chosen on the other seven runs, as in E1, removes 0.82 to 0.90 on seven runs and lands past the criterion on two to five of the eight. Seed 712 again takes a slice-1 edit for LEACE, INLP, and the probe, which keep 0.36 to 0.43 there with a choice made on the other runs; difference in means keeps 0.80, its choice on the others being a slice-2 edit that still works there. With the choice made at a spill of 0.01, as in E1, LEACE lands past the criterion on one run with about the same removal (0.84), while difference in means falls to 0.56.

Grading and where the answers go with the pooled fits¶

As in E2, on the held-out difference contexts, raw, at the configuration each fitter selected over the pooled fits alone.

With the pooled fits the edits still grade alike, and the anchored edit removes the largest share (0.96), with LEACE next at 0.94 and difference in means lowest at 0.72. LEACE lands a little further toward the target nullTarget null Where a removal should land: what an ideal predictor that had lost one op and nothing else would answer. It is the Bayes posterior over ops given the examples, with the removed op set to zero and the rest renormalized, so it still weighs the other ops by how well they fit. than the anchor (0.54 against 0.50), and difference in means less far (0.40). The answers go where they went in E2, except that difference in means leaves 0.21 on the answers of difference, against 0.04 for the anchor and 0.05 for LEACE.

Labels, write, and probe with the pooled fits¶

The label ladder stays flat for every pooled fit: from every label down to a fiftieth, the seed-mean removal moves by less than 0.02. At a fiftieth, LEACE, INLP, and the probe remove 0.83 to 0.91, above the 0.81 of the anchor with fixed labels at the same rate, and difference in means 0.70. So the number of labels is still not where the anchor has an edge.

Every pooled selection edits one slice, so its write equals its bound and E4 has nothing to measure. Slice 2 on most runs, and slice 1 on seed 712 for LEACE, INLP, and the probe.

A fresh probe still finds the op after every edit, as in E5: 0.64 to 0.74 at the query = of slice 3, and 0.70 to 0.82 at the example answers of slice 2. LEACE leaves the least at the answers, and the anchored edit the least at the query =.

Discussion¶

The preregistered comparison missed. Given role labels and a grid tuned on each run, every standard fitter that could be judged spilled no more than the anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. edit at matched removalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better. and removed more within the spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. criterion, mostly by editing the example answers alone. That is the result of record.

The comparison the anchor was built for restricts the fitters to fits that need no role labels, and it was added after the run. There, LEACELEACE Least-squares concept erasure (Belrose et al., 2023): the smallest affine edit, measured in the covariance-whitened norm, after which no linear probe can read the target. For one target it works with two directions: c, the covariance of the states with the target (for a binary target, the difference in means), and Σ⁻¹c, where Σ is the covariance of the states, which is the direction a least-squares probe reads. It reads how much of the concept a state holds along Σ⁻¹c and moves the state back along c until that reading is the mean, so afterwards the probe reads the same value for every state. The two directions differ, so the edit is oblique (not a perpendicular projection). and INLP remove about as much as the anchor and spill past the criterion on a few more runs, and difference in meansDifference in means The unit vector from the mean state of the other contexts to the mean state of difference contexts. The direction most steering work uses. The states are unit vectors, so their mean lies inside the sphere, but only the direction of the difference is used, and the edit puts each state back on the sphere. falls behind on both. So on this model the anchored edit matches the best post-hoc eraser we tried, and does better than the direction most steering work uses. We have not shown that it does better than LEACE.

The fitters had help the anchor did not. They saw every label at once, and each run chose among about a thousand configurations scored on that run. The anchor had one configuration, but it came from some thirty experiments at other seeds, so its search happened earlier. The configuration chosen on other runs in E1 is the closest this report comes to comparing the two kinds of search: for the fitters it works on most runs, with the runs over the criterion mostly an effect of choosing at the line, and on one run it misses because that run holds the op a slice earlier. One run in eight is too few to say how often a fixed recipe for a fitter would miss, though it fits the idea that placing the concept saves finding where it sits in each new model.

Placing the concept had costs: an extra training term, and a small drop in task score. What it offers that a post-hoc method lacks is a location chosen before training, and this experiment uses that only in the plainest way, one concept on one axis. Whether a chosen location helps more when several concepts are placed at once, when they are arranged in a designed subspace, or when a concept is harder to find after training, is open, and nothing here bears on it either way.

None of the edits, LEACE included, left a fresh probe at chance (E5). For the anchor that was expected, since we know part of the evidence lies off e₁. LEACE leaves about as much at the slice it edits, though its guarantee covers every linear direction there, so the reason that holds for the anchor does not carry over. E5 lists three candidate causes, and the one we would look at first is the step that puts each edited state back at unit length, which the guarantee does not cover. At the query =, the sans-diff modelsans-diff model A model trained with no loss on difference comes 0.58 of the way from the control to the target null, against 0.50 for the edited recipe of record, and misses in the same kind of way., which never learned the op, leaves a probe about as much as the edits do, so most of what a probe finds there may be what any model of these contexts holds.

The comparison covers one concept in one small model on a synthetic task, with eight runs per arm, and several readings depend on the choice of baseline (E6). The pooled comparison and the configuration chosen on other runs were added after the run, so they describe what we saw rather than test a prediction.

Method¶

The fits. Difference in meansDifference in means The unit vector from the mean state of the other contexts to the mean state of difference contexts. The direction most steering work uses. The states are unit vectors, so their mean lies inside the sphere, but only the direction of the difference is used, and the edit puts each state back on the sphere. takes the binary target only. The ridge penalty is fixed at 0.01, the value ex-2.2.1ex-2.2.1 On the D2.1 checkpoints, every fitted direction that removed at least half of red from the control cost the lines with no red 0.37 or more, an order of magnitude above the anchored edit. used; it is added to the unnormalized Gram matrix of the fit states, so with this many states it barely regularizes. That is fine here: with 280,000 states in 64 dimensions a least-squares probe has little room to overfit, and the penalty is there to keep the solve stable where the covariance is nearly flat. A strong penalty would turn the probe toward the difference in means, which the grid has already. LEACELEACE Least-squares concept erasure (Belrose et al., 2023): the smallest affine edit, measured in the covariance-whitened norm, after which no linear probe can read the target. For one target it works with two directions: c, the covariance of the states with the target (for a binary target, the difference in means), and Σ⁻¹c, where Σ is the covariance of the states, which is the direction a least-squares probe reads. It reads how much of the concept a state holds along Σ⁻¹c and moves the state back along c until that reading is the mean, so afterwards the probe reads the same value for every state. The two directions differ, so the edit is oblique (not a perpendicular projection). takes a small ridge on the covariance, since the stream is unit-norm and its covariance is close to rank-deficient. INLP fits the ridge probeRidge probe The normalized weight vector of a linear readout of the target, fitted by least squares with an ℓ₂ penalty. It finds what can be read off the state linearly., projects its direction out of the fit states, and repeats. When several slices are edited, each gets its own fit. The anchor uses e₁ at every slice, but a fitted direction of the control can differ from slice to slice. With sequential refit, the fit at slice ℓ is made on the fit setFit set 280,000 fresh contexts, 40,000 per op, drawn from the training generator of the run with its replacement op noise: about the size of the training corpus, and drawn the same way. Each comes with its generating op and its posterior over ops. The states are taken from the clean pass of the run at every slice and position. The fits are made from means and covariances accumulated over the set, so the states need not be held in memory at once. as it arrives under the edits at the slices before ℓ, which keeps an oblique eraser on the states it will see (ex-2.2.1 found it extrapolates poorly off them). Mean ablation replaces the coefficient with its mean over the fit contexts of the other ops.

The optimized directionOptimized direction A unit vector per edited slice, fitted by gradient descent on the control so that the edited answers on difference contexts move toward the target null while the answers on the other contexts stay where they were. It uses the target null, which no real pipeline has, so it is a diagnostic ceiling: about the best a rank-1 edit (an edit along one direction) of this control could do with the scoring in hand. INLP edits a span of several directions, so this ceiling does not bound it. COAST is its nearest published relative.. A unit vector per edited slice, initialized at the difference in means and fitted by Adam for a few hundred steps on the fit set. The loss is the KL divergence⁶6 KL divergence: how far one probability distribution is from another, zero when they match. from the target nullTarget null Where a removal should land: what an ideal predictor that had lost one op and nothing else would answer. It is the Bayes posterior over ops given the examples, with the removed op set to zero and the rest renormalized, so it still weighs the other ops by how well they fit. to the edited answers on difference contexts, plus a weight times the KL divergence from the clean answers to the edited ones on the other contexts, at full dose. The weight takes four values, and the dose and slice sets are crossed as for the others. Its configurations join the frontier like any other.

Grid size. About 500 configurations for difference in means, 1,000 for the probe and for LEACE, 1,500 for INLP, and 450 for the optimized direction, counting doses; each is one scoring pass, reading the logits at the query = (which predicts the answer) only. The whole grid is scored on the tuning setTuning set Another 14,000 fresh contexts from the same generator, drawn the same way., and again on the test setTest set The 14,000 held-out contexts of ex-2.2.21, 2,000 per op, so the anchored numbers need no recomputing. for the figure of E1, where nothing is chosen. A fit is made once per (target, site, slice, refit) and reused across doses and positions.

The baseline. On the tuning set, the median over fifteen coordinate axes other than e₁, drawn once with a fixed seed before the run and shared by every run. On the test set, fifteen permuted fits per configuration, at the bracketing configurations of each point. LEACE is homogeneous in its covariance vector, so a permuted fit is a full-strength oblique erasure along a reference direction. The coordinate-axis median applies the same operator form (slices, positions, dose, replacement) along each of the 63 axes other than e₁; it does not depend on the fitter, so it is computed once per form.

Comparisons. Each method against the anchor as two samples of eight runs: the difference of seed means with a 95% Welch t interval³, and Fisher's exact test² for counts of runs over the criterion.

Smoke test. Before the full run, the whole pipeline runs end to end on two stored runs outside the comparison: the control of ex-2.2.23 at seed 704 and the falloff run at the same seed. Every stage runs: the fits for every method and factor, both baselines, the selection rule, the test scoring, the label ladder, the optimized direction, the measures of E2 to E6, and the tables, figures, and print of this report, with the one pair standing in for the eight (an interval over one run a side means nothing, but the code that computes it runs). The grid is thinned so that each level of each factor is visited at least once, and each stage is timed per pass, which gives the cost of the full run. So a fault anywhere shows up before the costly part. The smoke test also gives two early readings: H4 on the anchoredAnchored The full edit along e₁, as published in fixed-effects. Nothing is refitted or retuned for this experiment. run, so that a fitter that can't find a placed concept shows up before the full run, and where a probe for the op scores best on the control.

The query operands join the sites. Before the freeze, whether the control holds the evidence for the op at the example answers, as the anchored model does, was not known: the block1-editblock1-edit Block 1 writes the evidence for the op onto e₁ at the example answers, and a later block reads it at the query =. experiment found that the control gathers the operand colors at each example answer in block 1, but it fitted no probe for the op on a control. The smoke test fitted one at every role and slice of the control at seed 704. It scored best at the two operands of the query (an area under the ROC curve of 0.95 at slice 3), above the query = (0.92) and the example answers (0.83). The query answer scored higher still (0.99), but it comes after the = that predicts the scored answer, so an edit there cannot change it, and it was left out. So the query operands joined the sites fitted, as the plan said they would.

Details fixed with the code. A sequential refit makes the fit at each slice under the full-dose edits at the slices before it, at every position, with zero replacement, and is reused across the doses, positions, and replacements scored with it; mean ablation under a refit uses the mean state of the other ops as it arrives under those edits. A permuted fit takes no sequential refit: it reuses the clean permuted fit at each slice, since a direction with no concept in it barely changes the states the next fit sees. The optimized direction acts at every position; its keep weight takes the values 0.3, 1, 3, and 10, and each search runs 300 Adam steps of 512 contexts at a learning rate of 0.01. With the graded target, the term toward the target null is weighted by the posterior on difference and the keep term by the rest. Its mean ablation is scored along the searched direction without searching again, and its permuted baseline uses five searches rather than fifteen, since each is a gradient search. A permuted search is no blank: each context keeps its own target null under the permutation, so the search still pulls the difference contexts among its labels toward the null, and the favoring rule takes the axis median for its removalRemoval The drop in expected exact match on difference under the edit, as a share of the way from the clean score to the target null. Higher is better.. E2, E4, and E5 read, for each method, the bracketing configuration of removal at the spillSpill The largest drop on any other op under the edit. Lower is better, and the selectivity criterion is 0.02. criterion that lies within the criterion. The label ladder of E3 leaves out the optimized direction, which reads the target null rather than labels; on the graded target an unlabelled difference context has a target of zero. The probe of E5 reads the query = and the example answers at each slice, on 7,000 test contexts split in half, and reports the area under the ROC curve. The write of E4 is measured on 2,000 test contexts of the other ops, with a role per position within an equation. The selection of H4 over the pooled fits and the example answers, and the pooled-only selection of H2, were run as a second pass after the main one, on the same tuning scores; it scored the test-set baselines of the configurations it chose that the main pass had not.

Left out. The same comparison for red on the D2.1 checkpoints, where ex-2.2.1 gave the fitters far less, is not run here: this report is about the inferred op.

Cost. No training. Each run takes about 6,000 scoring passes: the grid on the tuning and test sets, the axis baseline on the tuning set, the permuted fits, and the ladder. The anchored runs take the same for H4, so sixteen runs come to about 100,000 passes, plus the state passes over the fit set and for sequential refit, and the optimized direction. The plan estimated eight to ten L4-hours, about $15, with $25 as the ceiling. The run took L40S cards: the full run cost $15.80 and the second pass for the restricted selections $0.64, and with the smoke tests and the speed-ups between them the experiment came to $22.14. The passes added after the paper rounds on the results, E2 to E5 at the pooled selections and the probe on the sans-diff modelssans-diff model A model trained with no loss on difference comes 0.58 of the way from the control to the target null, against 0.50 for the edited recipe of record, and misses in the same kind of way., brought the total to $22.72.

References¶

ex-2.2.1

On the D2.1 checkpoints, every fitted direction that removed at least half of red from the control cost the lines with no red 0.37 or more, an order of magnitude above the anchored edit.

fixed-effects

The full scoring suite on the recipe of record at seeds 712 to 719: removal 0.88 at the full dose, spill below the criterion at every dose, graded drops, a landing of 0.50, and a write within the bound.

fixed confirmation

τ_ℓ fixed at fresh seeds 712 to 719 keeps every anchor; it is the recipe of record, and its checkpoints and dose scoring are stored.

anneal confirmation

Trained the controls at seeds 712 to 719 that this comparison fits on, sharing initialization, batches, and corpus with the anchored runs.

block1-edit

Block 1 writes the evidence for the op onto e₁ at the example answers, and a later block reads it at the query =.

mid-block-edit

Editing the mid-block states as well gains nothing and spills; in passing it shows the query = near zero alignment throughout.

reference-baseline

The median drop along the other coordinate axes of the run replaces the paired control as the baseline, and the published numbers barely move.

sans-diff model

A model trained with no loss on difference comes 0.58 of the way from the control to the target null, against 0.50 for the edited recipe of record, and misses in the same kind of way.

fixed-labels

Holding one label draw fixed for the run, as a real labelled set would be, keeps the anchor at every rate tried, but a few runs end with the edit removing much less of the op.

low-runs

In every run the evidence that an example fits difference lies partly on e₁ and partly off it, and the edit along e₁ never touches the off-axis part.

ex-2.2.21

The 14,000 held-out contexts, 2,000 per op, that every D2.2 score is taken on.