What the anchor follows at the = tokens¶
Every = is a place where the model has to infer the op from the examples before it, so it is where we expected an anchor that held the inferred op to follow the posterior. On the recipe, the anchor is nearly absent at every =. With a softer pool it reaches them, and there it grades with the examples that fit difference, each by how well it fits. Whether it also discounts the examples that do not fit, as the posterior does, is more than one seed per trial can say. Part of what the softer pool puts on the = is the same on every op.
The backlog item asks whether the anchor holds the op the model has inferred, which would make the alignment grade with the posterior on difference. Example-evidence found that at an example answer the anchor follows that one example, where the line itself shows the op, so the test has to move to the = tokens, where the answer is still to be predicted. This page reads the alignment at every = on the stored checkpoints of ex-2.2.23 and of the τ × λ_a sweep, with no new training.
Observations¶
- How much anchor the
=tokens hold (E1): on the recipe, α at the query=ofdifferencecontexts is at most 0.06 (the control reaches 0.05), against about 0.9 at a fitting example answer. On the sweep it rises with τ, to as much as 0.66 on the soft pools, and part of what it rises to is the same on every op. - What the rise follows (E2): on the trials where α rises along the context, the posterior before the
=accounts for up to 74% of its variance, and a sum of the one-example posteriors does about as well. The two come apart on the examples that do not fit: one lowers the posterior by 0.10 on average at a fixed count of fitting examples, and changes α by 0.00 (-0.02 to 0.02), which is within what a posterior-following α would show at this slope. Among the examples that fit, α grades with how specific the example is. - Where it sits by depth (E3): the part shared by every op starts at the
=embedding, is largest in the first slices, and fades with depth; the part that follows the examples appears after block 2 and stays through block 4.
Scope¶
This is a re-analysis of stored checkpoints, with no preregistration and no gate. It covers every run of ex-2.2.23 (the recipe at τ = 0.1: 24 anchored runs and 24 controls, at 200 and 400 epochs) and every trial of the τ × λ_a sweep (40 trials at 400 epochs, one seed each, τ from 0.01 to 1 on the no-emb and whole-line arms). The sweep trials are the only stored runs with an anchor at the = tokens, so E2 and E3 rest on the eleven trials with τ at or above 0.3, and within those on the six whose α rises along the context by at least 0.1. With one seed per trial, a difference between trials cannot be told from the seed.
The held-out set has 14,000 three-example contexts, 2,000 of them under difference. The fits of E2 use the difference contexts; the other ops' contexts give the level of α where the examples say nothing about difference.
The measurements¶
α is the alignment with e₁ (the cosine between a state and the anchored direction), averaged over slices 2 to 4, at the = of each example and at the query =. E3 shows it by slice.
Each = is compared with the evidence before it, on the noise model of the corpus: the posterior on difference given the examples before that = (1/7 at the first =, where nothing has been shown), and the number of those examples that fit, meaning the answer shown is one difference can give on that pair. The two agree closely on this stimulus, since the posterior climbs with each fitting example. They come apart in two places. An example that does not fit pulls the posterior down and leaves the count alone. And a count ignores how specific a fitting example is: one that also fits several other ops lifts the posterior from 1/7 to about 0.15, and one that fits difference alone lifts it to about 0.7 (the one-example posterior, as in example-evidence). A third summary, the sum of the one-example posteriors of the examples before the =, is graded like the posterior and ignores non-fitting examples like the count.
How much anchor the = tokens hold (E1)¶
Before asking what the alignment at the = follows, the question is whether there is any. The figure shows α at the first = and at the query = of difference contexts on every run, by the temperature of the pool.
How much anchor the = tokens hold, by pool temperature. On the held-out difference
contexts, α at the first = (open marks, where no example has been shown) and at the query =
(filled marks, after three), joined per run. Circles: the no-emb arm of the sweep; squares: its
whole-line arm; diamonds: the 400-epoch anchored runs of ex-2.2.23, at the recipe τ, spread sideways
so they can be told apart; grey: the controls of ex-2.2.23, drawn at the same τ.
On the recipe there is almost none. The 400-epoch anchored runs of ex-2.2.23 reach 0.03 (0.01 to 0.06) at the query =, and the controls 0.01 (-0.03 to 0.05), where a fitting example answer on the same runs has α near 0.9 (embedding-lean, E1). The sweep trials below τ ≈ 0.2 sit with them, apart from the four at the sharpest pools whose = embedding came onto e₁ (the sweep, E2); on those the = state carries the latched embedding rather than anything the context put there, and they are left out below. So on every run trained with the recipe pool, the = tokens hold too little anchor to follow anything.
From τ ≈ 0.3 the marks climb, as the sweep found, and they do so in two parts. The open marks rise too: at the first =, where no example has been shown, α reaches 0.22 (0.07 to 0.36) on the soft trials, and on the other ops' contexts it is the same, 0.22 (0.07 to 0.35). That part is a property of the = state that every context shares, which a soft pool rewards since the pull is met at a position every difference context has. The second part is the rise from the first = to the query =, which happens on difference contexts (0.13 (0.04 to 0.32)) and not on the others (-0.01 (-0.06 to 0.03)). That part follows the examples, and it is what E2 looks at, on the six trials where it is at least 0.1.
What the rise follows (E2)¶
On the rising trials, does α at a = follow the posterior on difference given the examples before it? The three summaries agree on most contexts, so the comparison needs the places where they differ. The left panel plots α against the count of fitting examples, one line per = index, so a later = with the same count is one that has seen more non-fitting examples, which lower the posterior by 0.10 on average. The right panel takes the second =, after a single example, against the one-example posterior of that example, from near zero (it does not fit) to 0.7 (it fits difference alone).
What α at the = follows, on the six rising trials. Left: α against the number of
examples before the = that fit difference, one line per = index (the second and third
example and the query), the mean over trials with the range as a band. Right: α at the second =
against the posterior given the first example alone, in bins; the first bin holds the examples that do
not fit, and the rest split the fitting examples by how specific they are. One thin line per trial,
the mean in bold. A bin with fewer than 20 contexts is left out.
Against the count, the three = indices lie close together: at the same number of fitting examples, the query = has about the same α as the second =, whatever came between. Adding a non-fitting example at a fixed count changes α by 0.00 (-0.02 to 0.02), where the posterior falls by 0.10. That looks like α ignoring the examples that do not fit, but the two numbers are on different scales. At the slope of α on the one-example posterior in the right panel, an α that followed the posterior would drop by only a few hundredths per non-fitting example, and on the two trials with the most anchor the observed drops are about that size. So on this stimulus the sum and the posterior are too close for one seed per trial to tell apart. Against the one-example posterior, α rises steadily across the fitting bins: an example that fits difference alone lifts α by 0.11 (0.05 to 0.23) more than one that fits several ops, which is more than the step from a non-fitting example to the loosest fit (0.04 (0.01 to 0.11)). So among the examples that fit, α does grade with how well. The middle bins are thin, with 133 of the 1,369 fitting first examples between the loosest and the most specific, so that slope rests on few contexts, though every trial shows it.
The table gives the comparison over every = with evidence before it, for the soft trials. The posterior, the count, and the sum of one-example posteriors account for about the same share of the variance on each trial, as they must when they agree on most contexts. The sum leads on four trials by a few hundredths to a tenth. On the two trials with the most anchor at the =, the posterior leads by about a tenth.
| arm | τ | α at the first = | rise to the query = | r², posterior | r², count | r², sum | Δα per non-fitting example | step, non-fit to loosest fit | grade, loosest to tightest fit |
|---|---|---|---|---|---|---|---|---|---|
no-emb | 0.31 | 0.07 | 0.16 | 0.56 | 0.65 | 0.66 | +0.00 | 0.02 | 0.07 |
no-emb | 0.35 | 0.12 | 0.04 | 0.10 | 0.15 | 0.15 | +0.00 | 0.01 | 0.01 |
no-emb | 0.37 | 0.19 | 0.07 | 0.15 | 0.21 | 0.22 | +0.01 | 0.00 | 0.02 |
no-emb | 0.46 | 0.26 | 0.12 | 0.35 | 0.45 | 0.46 | +0.02 | 0.02 | 0.05 |
no-emb | 0.49 | 0.09 | 0.06 | 0.08 | 0.13 | 0.14 | +0.01 | 0.02 | -0.01 |
whole | 0.52 | 0.19 | 0.22 | 0.50 | 0.63 | 0.65 | +0.02 | 0.05 | 0.06 |
no-emb | 0.57 | 0.12 | 0.04 | 0.05 | 0.08 | 0.08 | +0.01 | 0.00 | 0.01 |
no-emb | 0.71 | 0.34 | 0.32 | 0.74 | 0.62 | 0.63 | -0.02 | 0.11 | 0.23 |
no-emb | 0.75 | 0.34 | 0.17 | 0.73 | 0.64 | 0.63 | -0.02 | 0.03 | 0.13 |
no-emb | 0.93 | 0.36 | 0.09 | 0.25 | 0.31 | 0.31 | +0.01 | 0.01 | 0.04 |
whole | 1.00 | 0.33 | 0.14 | 0.53 | 0.57 | 0.58 | -0.01 | 0.01 | 0.09 |
The sweep trials with a soft pool (τ ≥ 0.3), by τ. α at the first = of
difference contexts and the rise from there to the query =; the share of the variance in α over
the = tokens with evidence before them (r²) accounted for by the posterior, by the count of fitting
examples, and by the sum of the one-example posteriors; the mean change in α when one more non-fitting
example is added at a fixed count (the posterior falls by 0.10); then at the second =,
the step in α from a first example that does not fit to the loosest fit, and the further rise from
there to the most specific fit. The rising trials (rise ≥ 0.1) are the ones E2 plots.
Where it sits by depth (E3)¶
E1 split the anchor at the = into a part every context shares and a part that follows the examples. If the two sat at different depths, a pull or an edit could address one without the other.
Where the anchor at the = sits by depth, on the six rising trials. α at the first =
and at the query =, on difference contexts and on the other ops' contexts, at each slice from
the embedding (emb) to the output of block 4. The mean over trials, with the range as a
band.
They do, in part. The shared part is there from the embedding slice, where α at a = is the e₁ component of the = embedding itself (the sweep found the syntax embeddings partway onto e₁ at a soft pool), and on the mean it is largest after block 1 and fades from there, to about half of its peak by block 4. The part that follows the examples is absent until block 2, grows through block 3, and holds at block 4. So the last two blocks carry the contextual part with the least of the shared one, and an edit at the early blocks would act on the shared part alone. The trials differ in the size of the shared part (the bands) and in some of the detail: on two of the six the shared part peaks at the embedding rather than after block 1, and the contextual part first shows at block 2 on two trials, at block 3 on three, and only at block 4 on one. What they share is the order: the shared part first and fading, the contextual part later and holding.
Discussion¶
The item asked whether the anchor grades with the posterior on the op. At the = tokens, the sites where the model has to infer it, the answer on the recipe is that there is nothing to grade: the pull is met at the example answers, and the = tokens are left about where the control has them. That agrees with ex-2.2.21, which found the anchor weak at the query =, and extends it to the = of every example.
That absence may be reasonable. To predict the answer at a =, the model only needs to attend back to wherever the op is represented, at the example answers or spread over several earlier roles. The state at the = itself need not hold the op, so the anchor at the example answers may already be where the op is read from.
A softer pool puts anchor on the = tokens, and some of it follows the examples. It grades with the per-example judgements that example-evidence found at the answers: each fitting example adds to α by how well it fits. Whether an example that does not fit also takes something away, as it does from the posterior, this stimulus can't resolve at one seed per trial, since the posterior moves little there. If the anchor turns out to sum rather than weigh, that would fit how the label trains: it is binary on the true op and treats a context with a misleading example the same as one without. The two trials with the most anchor at the = sit a little closer to the posterior than to the sum. A stimulus with more examples that do not fit, or more seeds, would separate the two.
The part every context shares is the larger of the two on most soft trials, and it is the price of the softer pool: a state the model can place on e₁ once, at a token every difference context has, meets the pooled pull more cheaply than a contextual feature does. The sweep found the spill of the edit growing with τ, and a component on the = state of every op is one place that spill would come from. It sits earlier in the stack than the contextual part, which is some help: an edit on the last two blocks would remove most of the contextual part and about half of the shared one. So on the stored runs, the anchor can be made to reach the sites where the op is inferred, but what it holds there grades with the examples that fit, and it brings a shared component along.