What the anchor follows at the example answers¶

The backlog item asks whether the anchor holds the op the model has inferred, which would make it grade with the posterior, rather than the binary label it was trained on. Ex-2.2.22 (E3) took a first look and found a rise with the posterior at the example answers, plus an unexplained dependence on the answer index. This page re-analyzes the same stored measurements, with no new runs.

Observations¶

• Each example on its own (E1): plotted against the posterior given that example alone, the lines for the three answer indices lie on top of each other. On every anchored condition, that posterior accounts for at least 89% of the variance in α (r²), against at most 48% for the posterior given the examples so far.
• What the earlier examples add (E2): fitted together with the evidence before that example, the earlier evidence gets a weight of at most 0.05, against at least 1.2 for the example itself. It shows most on answers that do not fit difference.
• The answer color alone (E3): the answer color, without its operands, predicts at most 9% of the variance, so α depends on how the answer relates to its operands.
• Predicting the next answer (E4): at the query the model does combine the examples. With three examples, the control and hinge score 0.88 where the ideal predictor scores 0.95, more than one example alone could support. With one or two examples it falls well short of the ideal predictor, even on k-mixed, which was trained on those counts.

Scope¶

This is a re-analysis of the stored by-count pass of ex-2.2.22 on its three-example held-out set, with no preregistration and no gate. It covers the hinge condition (six runs: three paired with ex-2.2.21 and three replicates), the whole-line condition (three runs), k-mixed (three runs, one left out as below), and the control (five runs). Only the 2,000 held-out difference contexts are used, and only their example answers: as ex-2.2.22 found, nearly every query answer has a posterior near 1, so there is too little spread to compare there. E4 also uses the held-out sets with 1 to 5 examples, for the task score at the query.

The k-mixed run at model seed 702 has no anchor at the example answers: α stays near the control even where the example fits difference. It also scores lowest on the task of all the runs here, as the runs that took the slow path through training did in ex-2.2.21. It is left out of the numbers below.

The measurements¶

α is the alignment with e₁ (the cosine between a state and the anchored direction), averaged over slices 2 to 4, at the answer of each example. Slice 1 shows the same pattern, more weakly, and the embedding shows none.

Each example answer is compared with three summaries of the evidence, all on the noise model of the corpus:

• the posterior on difference given the examples up to and including that one (the posterior so far, the one ex-2.2.22 used);
• the posterior given that example alone, from a uniform prior over the seven ops (the one-example posterior);
• the posterior given only the examples before it (the earlier posterior), which is 1/7 at the first example.

About two thirds of the examples in a difference context fit it, meaning the answer shown is one difference can give on that pair. The rest come from replacement op noise or a random color. A single example can lift the posterior on difference from 1/7 to about 0.7 at most, since other ops can give the same answer on some pairs. An example that does not fit has a one-example posterior near zero.

Each example on its own (E1)¶

If α followed the posterior so far, answers with the same posterior would have the same α whatever their position. The left panel below repeats the comparison of ex-2.2.22 on hinge, and the right panel plots the same answers against the one-example posterior.

On the left the three lines are offset, as in ex-2.2.22: at a posterior between 0.6 and 0.8, α at the first answer is about as high as it gets, and at the third it is about a third lower. On the right they lie on top of each other. The seed bands are narrow on both sides. The whiskers are not: at the same posterior so far, α spreads over most of its range from one context to the next, and against the one-example posterior that spread shrinks to a few tenths or less. An answer whose example fits difference has α near 0.88 at every index, and one that does not fit stays low.

That explains the offset. At the first answer, a middling posterior means one example that fits difference and also fits another op. At the third, it usually means a mix of examples that fit and examples that do not, and α then depends on which kind the latest one is.

The table gives the same comparison as variance explained, for each anchored condition. The share of the examples so far that fit, a summary that weights every example equally, does a little better than the posterior so far and much worse than the one-example posterior.

What the earlier examples add (E2)¶

The one-example posterior leaves some variance unexplained, so the earlier examples might still add something. A least-squares fit of α on the one-example posterior and the earlier posterior together gives each a weight; both are on the same scale, from 0 to 1.

The earlier examples get a weight of a few hundredths, against more than one for the example itself. They show most on the answers that do not fit: there α is a little higher when the earlier examples favor difference than when they do not (0.16 against 0.09 on hinge). On answers that fit, the earlier examples make almost no difference.

The answer color alone (E3)¶

The one-example posterior depends on the operands as well as the answer. If α could be predicted from the answer color alone, the anchor would be closer to a property of the answer token. Averaged by answer color on half of the contexts and tested on the other half, the answer color accounts for 0.09 (0.06–0.14) of the variance on hinge (held-out R²), and similar on the other conditions. So the model places the state at an example answer on e₁ according to how that answer relates to its operands.

Predicting the next answer (E4)¶

The anchor follows each example on its own, but the next-token loss asks the model to predict each example answer at its =, which means combining the examples before it. How well does it do that? The by-count pass scored the query on held-out sets with 1 to 5 examples. The query after one example asks for the answer a second example would show (without the noise an example can carry), and the query after two asks for the third. So these scores stand in for how well the model predicts the answers of the later examples. The control and hinge were trained on three examples only, so for them the other counts are outside training; k-mixed was trained on 1 to 5.

The ideal predictor gets an expected exact matchExpected exact match The task score, as the chance that a drawn answer agrees with a true one. With drawn answers, the correct answer to a line is spread over two or more colors, so we take the chance that an answer drawn from the model agrees with one drawn from the true distribution. Its ceiling is that same chance for the true distribution against itself (Σq²), below 1 on the ops that round. about equal to its posterior, so it follows the diagonal. With three examples, the count they were trained on, the control and hinge come close to it: 0.88 against 0.95 in the top bin on hinge. With one example every condition falls well short, at about 0.43 against 0.70, and k-mixed, which was trained on that count, does only a little better (0.48). With two examples the gap is wider still: 0.62 against 0.96 on hinge, and 0.68 on k-mixed. k-mixed stays below the diagonal at every count, reaching 0.69 with three examples. A single example lifts the posterior to about 0.7 at most, and with three examples the control and hinge score well above that, so at the query the model does combine the examples. The anchor at the example answers does not follow that combining.

Discussion¶

At the example answers the anchor marks examples that look like difference more than contexts that do. The rise with the posterior that ex-2.2.22 saw comes mostly from this: the more examples fit, the higher the posterior, and each fitting example has a high α of its own. So this measurement does not yet show the anchor holding an inferred op. The sites where that could show are the = tokens, where the model predicts an answer it has not yet seen. The next-token loss covers every one of them, so the model is asked to combine the examples so far at each example = as well as at the query =. The answer, one token later, is where the example just shown is in view, which is what α follows. Ex-2.2.21 found the anchor weak at the query =, and α at the example = tokens was not stored.

This fits how the label works. The pull is pooled over the positions of a whole difference context, so the model can meet it at positions of its choosing, and the answers of the examples that fit are where a single line shows the op most plainly. The answers that do not fit stay low, which a pooled pull allows.

It may also bear on the spill that ex-2.2.23 found. A per-example judgement of whether an answer fits difference given its operands would fire on examples of other ops whose answers happen to fit, and removing e₁ there would touch those ops. This re-analysis covers only difference contexts, so it does not test that.