Anneal confirmation¶

The soft pool scoutsoft pool scout The pool over slices 1 to 3 with \(\tau_\ell\) held at 0.1 keeps the anchor on all eight seeds with no early spill, and one seed rises late to near the criterion. found a candidate for the recipe of record: the pull pooled over slices 1 to 3, with the slice temperature \(\tau_\ell\) held at 0.1.¹1 Slice temperature: how sharply the pooled pull picks among slices. The pull is a softmin-weighted mix over slice and position pairs, and \(\tau_\ell\) sets how peaked the weights over slices are, as the pooling temperature \(\tau_p\) does over positions. It keeps the anchor and stays within the spill criterion at seeds 704 to 711. But on one or two seeds, spill rises late and ends near the criterion. The late spill scoutlate spill scout The late spill follows the anti anneal, and sharpening \(\tau_\ell\) to 0.025 over epochs 64 to 360 lowers it with no measured cost to removal or the task. then found that sharpening \(\tau_\ell\) further late in training, to 0.025, lowered that late rise at no measured cost.

Here the two moves become one. \(\tau_\ell\) falls from 0.1 at the start to 0.025 at epoch 400, smoothly on a log scale. So it is still close to 0.1 while the anchor settles, and reaches the halfway point (0.05) at epoch 200. We call this recipe τ_ℓ anneal.

This is the confirmation run for adopting τ_ℓ anneal as the recipe of record. It trains the recipe to the end at eight fresh seeds (712 to 719), with a control at each seed. It also trains it to epoch 64 at forty seeds (704 to 743), to count lost anchors.

Observations¶

• The anchor at fresh seeds (E1): every run keeps the anchor and passes the task gate. RemovalRemoval The drop on difference under the full edit at every position and slice, as a share of the way from the clean score to the target null. Higher is better, and a run below 0.5 has lost the anchor. varies more across seeds than under τ_ℓ fixed, and about as much as under falloff. One seed is a little below the lowest falloff run.
• Spill at the end and along training (E2)Spill The largest drop on any other op, at the dose where it is largest. Lower is better, and a run above 0.02 is over the criterion.: no run ends over the criterion. One seed rises late to the line, and the same seed goes over under the edit at block 1 alone. Two seeds have brief bumps over the line in the first hundred epochs, which clear within 64 epochs. The soft pool scoutsoft pool scout The pool over slices 1 to 3 with \(\tau_\ell\) held at 0.1 keeps the anchor on all eight seeds with no early spill, and one seed rises late to near the criterion. saw none under τ_ℓ fixed at the same checkpoints.
• Lost anchors at forty seeds (E3): the callCall The early verdict on whether a run kept the anchor, from mean α at the example answers at slice 3 over epochs 44 to 64. A run below 0.15 is called lost, and one from 0.28 kept. A run in between is finished and measured. finds all forty runs kept, each well clear of the band where the call is unsure.

Scope and method¶

This is an exploratory study, with no preregistration and no gate. We set four checks before the runs. They guide the choice of recipe without deciding it, so a recipe that misses one narrowly can still be the better choice:

1. Every run at seeds 712 to 719 keeps the anchor (removalRemoval The drop on difference under the full edit at every position and slice, as a share of the way from the clean score to the target null. Higher is better, and a run below 0.5 has lost the anchor. at least 0.5).
2. No run at those seeds ends over the spillSpill The largest drop on any other op, at the dose where it is largest. Lower is better, and a run above 0.02 is over the criterion. criterion of 0.02.
3. Removal is no lower than that of falloff at seeds 704 to 711. The task passes the gate of the D2.2 design against the control at the same seed: a mean change over ops of at least −0.02, and at least −0.05 on any one op.
4. At most one run of the forty loses the anchor. That is about as few as step, soft τ (none of forty) and falloff (one) lost in the loss-rate screenloss-rate screen A call from α at the example answers over epochs 44 to 64 sorts lost anchors from kept ones; at forty seeds falloff lost one and step, soft τ none., and forty seeds can't tell those two counts apart.

τ_ℓ anneal is τ_ℓ fixed from the soft pool scoutsoft pool scout The pool over slices 1 to 3 with \(\tau_\ell\) held at 0.1 keeps the anchor on all eight seeds with no early spill, and one seed rises late to near the criterion. (in full, τ_ℓ 0.1, τ_ℓ fixed) with one change: \(\tau_\ell\) moves from 0.1 to 0.025 over epochs 0 to 400, on a minimum-jerk curve in log \(\tau_\ell\).²2 Minimum-jerk: a smooth S-shaped path that starts and ends at rest, so the change is slow at both ends and fastest in the middle. The position temperature \(\tau_p\) still sharpens from 1 to 0.1 by epoch 40, the anchor weight still warms up over the first 40 epochs, and the anti weight still anneals to the per-slice holds of falloff by epoch 360.

The forty runs of the count use the same seeds as the falloff and step, soft τ runs of the loss-rate screen. So each run pairs with a run of each of those recipes by initialization, batches, and label draws. Each run stops at epoch 64 for its callCall The early verdict on whether a run kept the anchor, from mean α at the example answers at slice 3 over epochs 44 to 64. A run below 0.15 is called lost, and one from 0.28 kept. A run in between is finished and measured., made as in that screen.

The runs at 712 to 719 continue from those stops to epoch 400, keeping a checkpoint every 32 epochs. Each control trains for the full schedule at the same seed, with no pull and no anti term. All runs trained and were measured on L4 GPUs.

The stored recipes ran at seeds 704 to 711, and τ_ℓ anneal at 712 to 719. So a difference between them includes the variation between two draws of eight seeds. Under falloff and step, soft τ, removal varies by about 0.15 across seeds, with one or two low seeds in eight. A difference in the seed mean smaller than a few hundredths is within that variation.

The measurements¶

Every edit below is the e₁ edit. Each score is the drop under that edit, less the median drop under the same edit along each of the other 63 coordinate axes of the run (the rule of the D2.2 design).

Removal

The drop on difference under the full edit at every position and slice, as a share of the way from the clean score to the target null. Higher is better, and a run below 0.5 has lost the anchor.

Spill

The largest drop on any other op, at the dose where it is largest. Lower is better, and a run above 0.02 is over the criterion.

Spill along training

Spill at the full dose, measured on each checkpoint a run kept (epochs 16, 32, and 64, then every 32 epochs) and at the end.

Task change

The clean score per op, less that of the control at the seed.

Call

The early verdict on whether a run kept the anchor, from mean α at the example answers at slice 3 over epochs 44 to 64. A run below 0.15 is called lost, and one from 0.28 kept. A run in between is finished and measured.

The anchor at fresh seeds (E1)¶

Does every run at the fresh seeds keep the anchor, and are removalRemoval The drop on difference under the full edit at every position and slice, as a share of the way from the clean score to the target null. Higher is better, and a run below 0.5 has lost the anchor. and the task as good as under falloff?

Every run keeps the anchor, and every run passes the task gate. The task stays within about a hundredth of the control on the mean over ops. The worst single op is darken at seed 717 (−0.036), against a gate of −0.05. falloff has one run that misses the gate on one op.

Removal is close to that of falloff on the seed mean, and below that of τ_ℓ fixed. Most of the difference comes from two seeds, 719 and 718, which remove about as little as the lowest runs of the step recipes (falloff and step, soft τ). The lower of the two, seed 719 at 0.74, is the only run below the lowest falloff run. It is still well clear of the line where a run counts as lost. The other six spread from 0.83 to 0.92.

So removal under τ_ℓ anneal varies across seeds the way it does under the step recipes, while under τ_ℓ fixed it was unusually even at its eight seeds. We can't tell from this whether the anneal or the seeds make the difference, since the two recipes never ran at the same seeds.

The late spill scoutlate spill scout The late spill follows the anti anneal, and sharpening \(\tau_\ell\) to 0.025 over epochs 64 to 360 lowers it with no measured cost to removal or the task. found that sharpening \(\tau_\ell\) late kept removal where it was, seed by seed, which points to the seeds. But there the sharpening ran over epochs 64 to 360, carried on from stored epoch-64 states, where here it runs over the whole schedule.

Spill at the end and along training (E2)¶

Does any run end over the spillSpill The largest drop on any other op, at the dose where it is largest. Lower is better, and a run above 0.02 is over the criterion. criterion, and does the late rise of the soft pool scoutsoft pool scout The pool over slices 1 to 3 with \(\tau_\ell\) held at 0.1 keeps the anchor on all eight seeds with no early spill, and one seed rises late to near the criterion. show up along training? The figure shows the spill of each seed along training. Beneath it are the clean score of the op the spill falls on, and the two schedules that move late.

No run ends over the criterion, and one ends at it. Seed 713, after an early bump, stays near zero until about epoch 300, then rises to 0.020 at the end. That is 0.0002 under the line, less than a change of GPU moved this score in the late spill scoutlate spill scout The late spill follows the anti anneal, and sharpening \(\tau_\ell\) to 0.025 over epochs 64 to 360 lowers it with no measured cost to removal or the task. (up to 0.003). So we count it as at the line rather than under it. The other seven end below or near zero.

This is the late kind of spill that the soft pool scout saw on one or two seeds in eight, at about the same size and on the same op (mix, as on seed 707 under τ_ℓ fixed). So the slow anneal does not remove it.

Every seed is still learning mix over the whole run, which fits the earlier finding, quoted in the late spill scout, that the late kind falls on ops a run is still learning. But seed 713 learns it no later than the others, so when mix is learned does not say why this seed rises and the others don't.

Early on, two seeds have brief bumps over the line. Seed 714 reaches 0.08 at epoch 32, while the position pool sharpens. It is still at the line (0.020) at the next checkpoint, and near zero by the one after. Seed 713 reaches 0.029 at epoch 96, and is near zero again by 128.

These bumps are the early kind of spill. The soft pool scout scored τ_ℓ fixed at seeds 704 to 711 on the same early checkpoints (16, 32, 64, and 128) and saw no spill above 0.003 there. By epoch 32 the anneal has moved \(\tau_\ell\) by about one percent, so the bumps do not come from the anneal.

Two runs of eight against none of eight can't tell whether the two draws of seeds differ here or the bumps are simply uncommon. The stored runs of the step recipes were not scored this densely, so we don't know how common such bumps are under them.

Under the edit at block 1 alone, one run goes over the criterion (713, at 0.028). The run that rises late is also the one that spills at block 1. That fits what recipe-comparerecipe-compare The spill of the pool with a soft slice temperature enters at block 1, the one slice the pool pulls and the step recipes leave alone. found under the pool: the spill enters with the edit at block 1. But one run of every recipe goes over at block 1 (table in E1), the step recipes included, so the block-1 spill on its own does not single out the pool.

Lost anchors at forty seeds (E3)¶

How many of the forty runs lose the anchor by the callCall The early verdict on whether a run kept the anchor, from mean α at the example answers at slice 3 over epochs 44 to 64. A run below 0.15 is called lost, and one from 0.28 kept. A run in between is finished and measured. at epoch 64, and how does that compare with falloff and step, soft τ at the same seeds?

The call finds all forty runs kept. The lowest window mean (mean αAlignment How far a state points along e₁: the cosine between the state and e₁, which for a unit-length state is its component along e₁; 1 on the anchor direction and 0 off it. over epochs 44 to 64) is seed 724 at 0.43, above the line where a run counts as kept, so no run needed finishing to settle its call.

With none lost of forty, the rate of lost anchors is below 7% (a one-sided 95% bound), the same bound as for step, soft τ in the loss-rate screenloss-rate screen A call from α at the example answers over epochs 44 to 64 sorts lost anchors from kept ones; at forty seeds falloff lost one and step, soft τ none.. The call is made at epoch 64. The eight runs that went on to epoch 400 kept the anchor there too. The other 32 runs were scored only for the call, so the count says nothing about their spillSpill The largest drop on any other op, at the dose where it is largest. Lower is better, and a run above 0.02 is over the criterion..

Seed by seed, the window means are above those of falloff on 38 of forty seeds and above those of step, soft τ on 35. A soft position temperature at the start, which τ_ℓ anneal shares with step, soft τ, may be what brings the example answers onto e₁ early on every seed. But τ_ℓ anneal is also above step, soft τ on most seeds, so the soft start is not the whole of it. The pool over three slices may add to it, but nothing here separates the two. The soft start was added so the pull would not latch onto a syntax token while it settles, and that is its main job; whether a sharp slice temperature now does that job on its own is not something this run tests.

Discussion¶

τ_ℓ anneal meets three of the four checks outright. Every run at the fresh seeds keeps the anchor (check 1), none ends over the spillSpill The largest drop on any other op, at the dose where it is largest. Lower is better, and a run above 0.02 is over the criterion. criterion though one ends at it (check 2), and the count of lost anchors is the lowest a count of forty can give (check 4).

Check 3 has two halves, and the task half passes on every run. On the removalRemoval The drop on difference under the full edit at every position and slice, as a share of the way from the clean score to the target null. Higher is better, and a run below 0.5 has lost the anchor. half, τ_ℓ anneal is a little below falloff on the seed mean, by less than two draws of eight seeds can differ (Scope and method). So that check is met within the seed spread rather than outright.

The late rise is still there on one seed in eight, at about the size the soft pool scoutsoft pool scout The pool over slices 1 to 3 with \(\tau_\ell\) held at 0.1 keeps the anchor on all eight seeds with no early spill, and one seed rises late to near the criterion. saw. So the slow anneal of \(\tau_\ell\) does not remove the late kind of spill; at most it keeps it at the line here.

The late spill scoutlate spill scout The late spill follows the anti anneal, and sharpening \(\tau_\ell\) to 0.025 over epochs 64 to 360 lowers it with no measured cost to removal or the task. found the effect of the late sharpening to be about the size of the difference between two GPUs, so a confirmation at eight seeds can't tell whether it helps. What the run shows is no cost we can separate from the seed draw: removal is lower than under τ_ℓ fixed on the mean, and E1 can't say whether the anneal or the seeds did that.

Two things favor the pool recipe over falloff. It chooses the slice within a range set by the architecture (slices 1 to \(L-1\)), and its early callCall The early verdict on whether a run kept the anchor, from mean α at the example answers at slice 3 over epochs 44 to 64. A run below 0.15 is called lost, and one from 0.28 kept. A run in between is finished and measured. is never unsure, where falloff stayed in the unsure band on three seeds of forty. Its count of lost anchors, none of forty against one, is a tie at this many seeds.

Against the pool recipe, one seed in eight ends at the spill line, while the eight stored runs of falloff end at most at 0.014. That is one run, set against the seed draw and the GPU shift, so it is a small mark rather than a difference we can rely on. Under the block-1 edit it goes over on one seed, as one run of every recipe does, and its removal varies across seeds as under the step recipes.

The early bumps on two seeds are the other mark, and the larger one goes well over the line. Under τ_ℓ fixed the soft pool scout saw none at the same checkpoints, but at epoch 32 the anneal has moved \(\tau_\ell\) by about one percent, so the two recipes are nearly the same there and the difference is most likely between the two draws of seeds. Both bumps cleared within 64 epochs, and the step recipes were never scored this early, so we can't say whether they bump too.

References¶

soft pool scout

The pool over slices 1 to 3 with \(\tau_\ell\) held at 0.1 keeps the anchor on all eight seeds with no early spill, and one seed rises late to near the criterion.

late spill scout

The late spill follows the anti anneal, and sharpening \(\tau_\ell\) to 0.025 over epochs 64 to 360 lowers it with no measured cost to removal or the task.

loss-rate screen

A call from α at the example answers over epochs 44 to 64 sorts lost anchors from kept ones; at forty seeds falloff lost one and step, soft τ none.

recipe-compare

The spill of the pool with a soft slice temperature enters at block 1, the one slice the pool pulls and the step recipes leave alone.