Ex 2.2.20: A high-rate head start before the recipe schedule¶

Findings¶

• The head start keeps most of the skill (H1) — miss. The head-start runs fall short of the 400-epoch runs, inside the partial band, but all three HSV ops fall short by more than their tolerance.
• The head start beats the plain schedule (H2) — miss. The head start scores above the plain 200-epoch run in 1 of the three seeds, but below it on average. We keep the plain schedule.
• The HSV ops come earlier (H3) — miss. They came later than in the plain run in two of the three seeds.
• Where the first cycle leaves off (E1) — the head start leads the plain run at epoch 50, but the second warmup takes the lead away within 5 epochs.
• Calibration (E2) — the head start is about as well calibrated as the plain 200-epoch run.
• Op confusion (E3) — the head start puts more mass off the diagonal in the HSV rows than the plain 200-epoch run does.
• A lower second peak (E4) — misses both rules by a wider margin.

Why this experiment¶

Ex-2.2.19 adopted 200 epochs for the seven-op set (no-four), half the length of the recipe, as a partial pass. On three fresh seeds the 200-epoch runs fell short of the 400-epoch runs by a little more than we would like, and nearly all of the shortfall was in the three HSV ops.

A short run at a high rate gets a long way quickly. The 50-epoch run of ex-2.2.19 at a high peak learning rate ended with more skill than any of the 200- and 400-epoch runs had at epoch 50. But it ends before the HSV ops are learned, which in the 200-epoch runs happened later in the run.

So we try both in one run: 50 epochs on the schedule of that short run, then 150 epochs on the recipe schedule from where it leaves off. If the second cycle builds on the first, the run should end closer to the 400-epoch runs than a plain 200-epoch run does, at the same cost.

Parameters¶

Each run follows the 200-epoch recipe of ex-2.2.19 with a different learning-rate schedule. The first cycle warms up over 5 epochs to 0.00562 and follows a cosine down to 1% of 0.00316 at epoch 50. The second warms up from there over 5 epochs to 0.00316, and follows a cosine down to 1% of that at epoch 200.

The second cycle is the recipe schedule of a 150-epoch run, step for step.¹ The first cycle is the schedule of the ex-2.2.19 run at 0.00562, except that it ends where the second warmup starts. The optimizer state is kept from one cycle to the next, though resetting it should make no measurable difference.²

The schedule differs from the plain one: it has a higher peak for the first 50 epochs, a second warmup, and a final anneal 150 epochs long instead of 200. These runs test the three together, so they cannot tell which of them made a difference.

A second head-start condition peaks at 0.0021 in the second cycle, about the highest rate at which a plain 200-epoch run of ex-2.2.19 learned the HSV ops (see H3, below). So it spends longer near those rates, and less time above them, than the first head start. From about epoch 80 on, its rate is close to that of the plain 200-epoch run, so the two differ mostly in the first 80 epochs.

Held-out expected exact match (EEM) is the probability that an answer drawn from the model at the query = is a correct answer of the true op. The Bayes ceiling is the same score for an ideal predictor. Skill is EEM rescaled so that a uniform guess scores 0 and the ceiling scores 1.

A run falls short of another by how much lower its EEM is, compared between runs that share a model seed and averaged over seeds. The hypotheses are scored on seeds 601–603, as H1 of ex-2.2.19 was. Seed 600 is shown beside them: the observation behind this experiment came from its runs, so it could flatter the head start.

The head start keeps most of the skill (H1)¶ ✗ Miss

What we expect. The head-start runs would fall short of the 400-epoch runs by at most 0.015 on average, and no op would fall short by more than its tolerance on average. That would be a pass. A shortfall between 0.015 and 0.03 would be a partial pass, and one beyond 0.03, or an op beyond its tolerance, would be a miss. This is the H1 rule of ex-2.2.19, with the same tolerances.

What we saw. The head-start runs fell short of the 400-epoch runs by a little more than the plain 200-epoch runs did, inside the partial band. The shortfall is in the HSV ops: each of the three falls short by more than its tolerance, where none did in the plain runs.

At seed 600 the head start scored above the 400-epoch run, so this seed did flatter the head start.

The head start beats the plain schedule (H2)¶ ✗ Miss

What we expect. The head-start run would score a higher EEM than the plain 200-epoch run in each of the three seeds, and so on average. That would be a pass. A higher mean with a lower EEM in one or two seeds would be a partial pass, and a mean gain at or below zero would be a miss.

A miss would mean the second cycle kept nothing of the first, or lost it in the second warmup. H2 checks only the direction of the gain: the whole gap to the 400-epoch runs is small, so H1 covers its size.

Which schedule we adopt. H1 and H2 score the head start with the recipe peak, and E4 scores the one with a second peak of 0.0021 by the same rules. If both hypotheses pass for either schedule, the 200-epoch runs that follow will use that schedule. If both schedules pass, we take the one with the higher mean EEM over seeds 601–603. Otherwise we keep the plain schedule.

What we saw. The head start scored above the plain 200-epoch run in 1 of the three seeds, and below it on average. The ops that lose most are the HSV ops again.

The HSV ops come earlier (H3)¶ ✗ Miss

What we expect. In the plain 200-epoch runs, the mean skill of the HSV ops first passed 0.5 between epochs 80 and 112. We expect the head-start runs to pass it about 25 epochs earlier, paired by seed (a gap estimated from the skill curves of ex-2.2.19).

The runs of ex-2.2.19 already suggest these ops do not wait for the learning rate to fall to some level. The 400-epoch runs passed 0.5 at about the same epochs as the 200-epoch runs of the same seeds, but at learning rates higher than any rate at which a 200-epoch run passed it.

The 50- and 100-epoch runs fell through those rates to their floor, and their HSV skill never passed 0.5. That includes the 50-epoch run at the higher peak rate, which is the first cycle of the head-start schedule.

So this schedule can separate elapsed time from learning progress. A rise at about the same epoch as in the plain run would say these ops wait for a number of epochs, whatever the model has learned by then. An earlier rise, as we expect, would say they build on the progress of the first cycle.

A later rise is possible too: the 200-epoch run of ex-2.2.19 at the higher peak rate passed 0.5 later than any 200-epoch run at the recipe peak, so a high early rate may hold these ops back. The predicted gap is several times the logging interval of 4 epochs, so the logged curves can tell these outcomes apart. There is no gate, since no decision hangs on it.

What we saw. The HSV ops came later in the head-start runs in two of the three seeds, and earlier in the third. The curves show the first cycle lifting the other ops well above the plain run by epoch 50, with the HSV ops still near zero.

Where the first cycle leaves off (E1)¶

Exploratory, with no prediction: how much of a head start the second cycle receives, and how much of it remains after the second warmup. We show the skill of the head-start runs at epoch 50, overall and per op, beside the plain runs at the same epoch and the 50-epoch ex-2.2.19 run at the higher peak rate (seed 600 only).

The head start leads the plain run at epoch 50 over all ops, while the HSV skill is still low in both. The second warmup takes the lead away by epoch 55. The head start with a second peak of 0.0021 keeps more of its lead, but ends lower all the same (E4).

Calibration (E2)¶

Exploratory, with no prediction. We show the calibration KL of each run beside its EEM: the KL divergence³ from the Bayes answer distribution to the answer distribution of the model. In ex-2.2.19 the 200- and 400-epoch runs were equally well calibrated, so a head start that buys EEM at some cost to calibration would show here.

The calibration KL of both head starts overlaps that of the plain 200-epoch runs. Among the 200-epoch runs, KL falls as EEM rises, so the runs that score lower are also less well calibrated.

Op confusion (E3)¶

Exploratory, with no prediction. We show the op confusion matrix of each schedule, as in ex-2.2.19. For confident contexts of each true op, it gives the probability mass the model puts on the answers of each op, averaged over seeds 601–603.

In ex-2.2.19 the 200-epoch runs put more mass off the diagonal than the 400-epoch runs, mostly in the HSV rows. This shows whether the head start moves that mass back.

The head start puts more of that mass off the diagonal in the HSV rows than the plain 200-epoch run, and the head start with a second peak of 0.0021 puts more still. In the other rows the four schedules are about the same.

A lower second peak (E4)¶

Exploratory, with no prediction. We score the head start with a second peak of 0.0021 by the rules of H1 and H2. As in H3, we also show the epoch at which its HSV skill first passes 0.5, beside the same epoch for the other head start.

Its results sit beside the other head start in the H1, H2, and H3 figures and tables. It misses both H1 and H2. It falls short of the 400-epoch runs by more than the other head start, at the edge of the partial band (0.0300), with the three HSV ops each beyond their tolerance. It scores below the plain 200-epoch run in all three seeds.

Its HSV ops came later than in the plain run in two seeds, and earlier in the third.

Discussion¶

It seems we should keep the plain schedule from ex-2.2.19.

The head start does what the 50-epoch run of ex-2.2.19 suggested, while it lasts: by the end of the first cycle the model has learned most ops except the HSV ones. The second warmup then takes the lead away (E1). The second cycle seems to relearn those ops rather than build on the first.

The HSV ops came no earlier (H3), and the head-start runs ended with more of their probability mass on the answers of the wrong op (E3).

The lower second peak was meant to give those ops more time at the rates where the plain runs learned them, yet it ended furthest short (E4). So time near those rates does not seem to be what the HSV ops wait for, although the schedule changes three things at once and the three seeds disagree about the timing.

We don't know whether the second warmup itself costs the lead. A schedule that hands over from the first cycle without a second warmup may answer that, although we did already try schedules with plateaus in ex-2.2.17.

Method¶

Recipe. The unanchored d64-L4 model with an untied readout and the newline mask, as in ex-2.2.19, on the no-four corpus condition that ex-2.2.18 built (k3-r0.3). The runs differ from the plain 200-epoch runs of ex-2.2.19 in the learning-rate schedule alone.

Measurements. EEM, ceiling, floor, and calibration KL on the 2,000 held-out contexts per op, with the evaluation of ex-2.2.18. Skill curves are logged at about 50 points per run on 200 held-out contexts per op, every 4 epochs at 200 epochs, so an epoch of first passing is known to within that.

Cost. The new runs cost about $1.04 in all, scaled from the cost of an ex-2.2.18 run.

Per-op tolerances. As in ex-2.2.19: the seed range of the gap of each op over the three ex-2.2.17 runs, or 0.03, whichever is larger.