import%20marimo%0A%0A__generated_with%20%3D%20%220.23.16%22%0Aapp%20%3D%20marimo.App(%0A%20%20%20%20width%3D%22medium%22%2C%0A%20%20%20%20app_title%3D%22Ex%202.2.1%3A%20suppressing%20red%20in%20the%20anchored%20transformer%22%2C%0A%20%20%20%20css_file%3D%22..%2F..%2Freport.css%22%2C%0A%20%20%20%20auto_download%3D%5B%22html%22%5D%2C%0A)%0A%0Awith%20app.setup(hide_code%3DTrue)%3A%0A%20%20%20%20import%20marimo%20as%20mo%0A%0A%20%20%20%20%23%20Once%20results%20exist%2C%20the%20design%20constants%20and%20refs%20come%20from%20%60experiment.py%60%0A%20%20%20%20%23%20beside%20this%20notebook%20(Marimo%20puts%20the%20notebook%20directory%20on%20sys.path)%3B%20the%0A%20%20%20%20%23%20skeleton%20quotes%20the%20frozen%20gates%20in%20prose%20and%20that%20module%20carries%20the%20same%0A%20%20%20%20%23%20numbers%20with%20each%20gate's%20wording%20in%20its%20docstring.%0A%20%20%20%20from%20mini.reports%20import%20report_bundle%2C%20use_publisher%0A%0A%20%20%20%20use_publisher(report_bundle(__file__))%0A%0A%20%20%20%20%23%20A%20cell%20renders%20its%20last%20expression%2C%20and%20a%20trailing%20docstring%20is%20one.%0A%20%20%20%20None%0A%0A%0A%40app.cell(hide_code%3DTrue)%0Adef%20_()%3A%0A%20%20%20%20mo.md(r%22%22%22%0A%20%20%20%20%23%20Ex%202.2.1%3A%20suppressing%20*red*%20in%20the%20anchored%20transformer%0A%0A%20%20%20%20%2F%2F%2F%20tip%20%7C%0A%20%20%20%20%3C!--%20tl%3Bdr%20--%3E%0A%20%20%20%20The%20first%20intervention%20on%20an%20anchored%20transformer.%20We%20take%20the%20D2.1%20checkpoints%20and%20remove%20the%20anchor%20axis%20from%20the%20residual%20stream%2C%20then%20score%20completion%20accuracy%20on%20lines%20with%20a%20red%20operand%20and%20on%20lines%20without.%20Five%20questions%3A%20does%20suppression%20bite%2C%20is%20it%20selective%2C%20does%20it%20grade%20with%20redness%2C%20does%20the%20edit%20on%20non-red%20lines%20stay%20within%20the%20bound%20the%20placed%20geometry%20sets%2C%20and%20does%20zeroing%20the%20weights%20that%20carry%20the%20axis%20do%20the%20same%20job%3F%0A%20%20%20%20%2F%2F%2F%0A%20%20%20%20%22%22%22)%0A%20%20%20%20return%0A%0A%0A%40app.cell(hide_code%3DTrue)%0Adef%20_()%3A%0A%20%20%20%20mo.md(r%22%22%22%0A%20%20%20%20%23%23%20Findings%0A%0A%20%20%20%20%2F%2F%2F%20admonition%20%7C%20TODO%0A%20%20%20%20Empty%20until%20the%20results%20land%3A%20one%20verdict%20line%20per%20hypothesis%2C%20with%20the%20deciding%20number%20and%20its%20gate%20inline.%0A%20%20%20%20%2F%2F%2F%0A%20%20%20%20%22%22%22)%0A%20%20%20%20return%0A%0A%0A%40app.cell(hide_code%3DTrue)%0Adef%20_()%3A%0A%20%20%20%20mo.md(r%22%22%22%0A%20%20%20%20%2F%2F%2F%20admonition%20%7C%20How%20to%20read%20this%20draft%0A%20%20%20%20This%20is%20a%20preregistration.%20The%20hypotheses%20and%20their%20gates%20below%20are%20frozen%20as%20of%20the%20commit%20that%20adds%20this%20file%3B%20results%20will%20replace%20the%20%60TODO%60%20placeholders%20in%20place%2C%20and%20anything%20conceived%20after%20seeing%20the%20data%20goes%20under%20%5BExploratory%20analyses%5D(%23exploratory-analyses)%2C%20marked%20as%20post%20hoc.%0A%0A%20%20%20%20While%20this%20draft%20was%20written%20we%20ran%20a%20one-seed%20smoke%20test%20of%20the%20operator%20library%20(seed%200%20of%20each%20condition)%2C%20to%20check%20the%20code%20path%20from%20checkpoint%20to%20score.%20The%20gates%20come%20from%20the%20clean%20statistics%20instead%3A%20the%20published%20alignment%20maps%2C%20the%20task-gate%20width%20the%20D2.1%20experiments%20used%2C%20and%20the%20reference%20values%20from%20M1.%20None%20of%20the%20smoke-test%20numbers%20are%20quoted%20here.%0A%0A%20%20%20%20One%20smoke-test%20observation%20is%20worth%20stating%20in%20advance.%20At%20one%20syntax%20position%2C%20the%20alignment%20arriving%20at%20the%20second%20slice%20under%20intervention%20sat%20above%20the%20clean%20envelope%20on%20that%20seed.%20We%20keep%20the%20deeper-slice%20clause%20of%20H4%20as%20the%20design%20states%20it%3B%20the%20nine-seed%20read%20will%20say%20how%20general%20that%20behavior%20is.%0A%20%20%20%20%2F%2F%2F%0A%0A%20%20%20%20%23%23%20Why%20this%20experiment%0A%0A%20%20%20%20D2.1%20anchored%20*red*%20on%20the%20first%20axis%20of%20the%20residual%20stream%2C%20and%20the%20geometry%20landed%20where%20we%20asked%3A%20red%20states%20aligned%2C%20the%20non-red%20bulk%20contained%2C%20grading%20roughly%20sensible%2C%20task%20accuracy%20unchanged.%20It%20never%20ran%20an%20intervention.%20The%20point%20of%20placing%20a%20concept%20is%20that%20removing%20it%20afterwards%20has%20a%20predictable%20effect%20and%20a%20side-effect%20we%20can%20bound%2C%20and%20we%20have%20not%20tested%20that%20claim%20in%20a%20transformer.%0A%0A%20%20%20%20The%20test%20uses%20checkpoints%20already%20in%20the%20store.%20Projecting%20e%E2%82%81%20out%20of%20the%20stream%20at%20inference%20time%20costs%20nothing%20to%20train%20and%20about%20a%20second%20per%20run%20to%20score%2C%20which%20makes%20it%20the%20most%20information%20per%20dollar%20in%20the%20%5BD2.2%20plan%5D(..%2Fd2.2%2Fdesign.md).%0A%0A%20%20%20%20It%20is%20also%20where%20the%20first%20three%20risks%20in%20that%20plan%20get%20their%20first%20data%3A%0A%20%20%20%20(a)%20that%20suppression%20does%20not%20bite%20even%20on%20*red*%2C%0A%20%20%20%20(b)%20that%20the%20bound%20is%20loose%20or%20wrong%20once%20later%20blocks%20sit%20between%20the%20write%20and%20the%20output%2C%20and%0A%20%20%20%20(c)%20that%20the%20response%20to%20suppression%20is%20undesigned.%20Only%20the%20first%20is%20retired%20here.%20The%20plan%20shares%20the%20second%20with%20the%20layer%20sweep%20and%20the%20third%20with%20the%20fallback%20experiment.%0A%20%20%20%20%3C!--%20REVIEW%3A%20was%20%22settles%20the%20first%20three%20risks%22.%20The%20design's%20risk%20table%20retires%20only%20the%20first%20here%20%E2%80%94%20the%20bound%20risk%20is%20%22suppress%20red%2C%20then%20layer%20sweep%22%2C%20and%20the%20undesigned-response%20risk%20is%20measured%20here%20and%20pinned%20by%20the%20fallback%20experiment.%20Verify%20against%20the%20risk%20table%20in%20%2Fdocs%2Fm2%2Fd2.2%2Fdesign.md.%20--%3E%0A%0A%0A%0A%20%20%20%20Three%20findings%20from%20the%20M1%20autoencoder%20result%20carry%20over%20here.%20Suppression%20there%20removed%20*red*%20almost%20completely%2C%20left%20orthogonal%20colors%20untouched%2C%20and%20degraded%20warm%20colors%20in%20proportion%20to%20their%20alignment%20with%20the%20axis.%20Those%20become%20H1%2C%20H2%2C%20and%20H3.%0A%0A%20%20%20%20The%20bound%20does%20not%20carry%20over%20as%20it%20stands.%20In%20the%20autoencoder%2C%20the%20decoder%20was%20a%20single%20linear%20readout%20from%20the%20point%20of%20intervention%2C%20so%20the%20geometry%20bounded%20the%20output%20directly.%20Here%20every%20later%20block%20sits%20between%20the%20write%20and%20the%20logits.%0A%0A%20%20%20%20So%20we%20state%20the%20bound%20as%20layer-local%2C%20following%20the%20%5Bkickoff%20lessons%5D(%2Ftodo%2Fscience%2Fd21-kickoff-carry-over-lessons.md)%3A%20the%20geometry%20bounds%20the%20immediate%20write%20at%20each%20intervened%20slice%20(H4).%20Behavior%20is%20then%20a%20separate%20prediction%2C%20scored%20by%20the%20task%20gate%20(H2).%0A%0A%20%20%20%20These%20checkpoints%20have%20no%20fallback%20term%2C%20so%20whatever%20a%20red%20line%20decodes%20to%20after%20suppression%20is%20the%20untrained%20response%20of%20the%20model%20itself.%20M1%20called%20that%20spoofing%2C%20and%20it%20is%20where%20the%20seed%20spread%20in%20ex-2.9.1%20came%20from.%20We%20measure%20it%20here%20without%20a%20gate%3B%20it%20is%20the%20reference%20the%20fallback%20experiment%20after%20this%20one%20has%20to%20improve%20on.%0A%20%20%20%20%22%22%22)%0A%20%20%20%20return%0A%0A%0A%40app.cell(hide_code%3DTrue)%0Adef%20_()%3A%0A%20%20%20%20mo.md(r%22%22%22%0A%20%20%20%20%2F%2F%2F%20details%20%7C%20Glossary%0A%20%20%20%20-%20**line**%20%E2%80%94%20one%20equation%2C%20%60c1%20%2B%20c2%20%3D%20answer%60%2C%20six%20word-level%20tokens.%20The%20probe%20set%20enumerates%20all%205%2C832%20closed%20lines%20of%20the%20216-color%20grid%3A%20every%20color%20as%20op1%20against%20each%20of%20its%2027%20closed%20partners.%0A%20%20%20%20-%20**residual%20stream**%20%E2%80%94%20the%20running%20vector%20each%20token%20carries%20through%20the%20model%2C%20which%20every%20block%20reads%20from%20and%20adds%20back%20into.%0A%20%20%20%20-%20**slice**%20(%24%5Cell%24)%20%E2%80%94%20a%20depth%20at%20which%20the%20residual%20stream%20is%20read%3A%20the%20embedding%2C%20plus%20the%20stream%20after%20each%20of%20the%20four%20blocks.%20Five%20slices.%0A%20%20%20%20-%20**alignment**%20(%24%5Calpha%24)%20%E2%80%94%20%24%5Ccos(h%2C%20e_1)%24%2C%20a%20state's%20cosine%20to%20the%20anchor%20axis.%20States%20are%20unit-norm%2C%20so%20it%20is%20the%20e%E2%82%81%20component%20itself.%0A%20%20%20%20-%20**dose**%20%E2%80%94%20the%20redness%20of%20a%20line%2C%20taken%20as%20the%20larger%20of%20the%20two%20operand%20rednesses%20(the%20label%20affinity%20from%20M1%2C%20%24r(1%20-%20g%2F2%20-%20b%2F2)%24%20on%20the%20unit%20cube).%20Either%20operand%20can%20carry%20the%20concept%2C%20since%20either%20could%20draw%20the%20label%20in%20ex-2.1.10.%0A%20%20%20%20-%20**red%20lines**%20%E2%80%94%20dose%20%E2%89%A5%200.8%3A%20an%20operand%20among%20the%20five%20reddest%20colors%20of%20the%20grid.%20365%20lines.%0A%20%20%20%20-%20**non-red%20lines**%20%E2%80%94%20dose%20%E2%89%A4%200.2.%201%2C689%20lines.%0A%20%20%20%20-%20**response**%20%E2%80%94%20the%20probability%20the%20model%20puts%20on%20the%20correct%20answer%2C%20read%20from%20the%20log-softmax%20at%20the%20%60%3D%60%20position.%20Accuracy%20is%20whether%20that%20answer%20is%20the%20argmax.%0A%20%20%20%20-%20**damage**%20%E2%80%94%20the%20clean%20response%20minus%20the%20intervened%20response%2C%20per%20line%2C%20in%20probability%20units.%0A%20%20%20%20-%20**write**%20%E2%80%94%20the%20rotation%20the%20operator%20applies%20to%20one%20state%2C%20in%20radians%3A%20the%20angle%20between%20the%20state%20as%20it%20arrived%20at%20the%20operator%20and%20the%20state%20the%20next%20block%20consumed.%0A%20%20%20%20-%20**clean**%20%E2%80%94%20the%20un-intervened%20forward%20pass%20of%20the%20same%20checkpoint.%0A%20%20%20%20%2F%2F%2F%0A%20%20%20%20%22%22%22)%0A%20%20%20%20return%0A%0A%0A%40app.cell(hide_code%3DTrue)%0Adef%20_()%3A%0A%20%20%20%20mo.md(r%22%22%22%0A%20%20%20%20%23%23%20Method%0A%0A%20%20%20%20No%20training.%20Every%20run%20is%20a%20published%20checkpoint%20of%20ex-2.1.10%2C%20scored%20on%20that%20experiment's%20published%20probe%20set%20through%20the%20eval%20contract%20in%20%5B%60sca.intervention%60%5D(%2Fsrc%2Fsca%2Fintervention.py)%3A%20the%20same%20lines%2C%20the%20same%20code%2C%20and%20the%20same%20statistics%20for%20every%20condition%2C%20arm%2C%20and%20baseline.%0A%0A%20%20%20%20%23%23%23%20The%20intervention%0A%0A%20%20%20%20Axis%20projection%20at%20full%20strength%3A%20at%20each%20slice%2C%20every%20state%20loses%20its%20e%E2%82%81%20component%20and%20is%20put%20back%20onto%20the%20sphere%2C%20since%20the%20next%20block%20expects%20a%20unit%20vector.%20That%20re-projection%20is%20part%20of%20the%20edit.%0A%0A%20%20%20%20For%20a%20state%20with%20alignment%20%24%5Calpha%24%2C%20the%20surviving%20components%20are%20rescaled%20by%20%241%2F%5Csqrt%7B1%20-%20%5Calpha%5E2%7D%24%2C%20and%20the%20whole%20edit%20amounts%20to%20a%20rotation%20by%20%24%5Carcsin%7C%5Calpha%7C%24.%20That%20formula%20depends%20only%20on%20the%20alignment%20the%20state%20arrived%20with%2C%20so%20we%20can%20compute%20the%20bound%20from%20the%20published%20alignment%20maps%20before%20the%20intervention%20runs.%0A%0A%20%20%20%20The%20primary%20intervention%20acts%20at%20all%20five%20slices%20and%20all%20six%20positions.%20The%20operator%20at%20slice%20%24%5Cell%24%20sees%20a%20state%20the%20blocks%20built%20from%20already-edited%20inputs%2C%20so%20only%20at%20the%20embedding%20does%20it%20see%20the%20clean%20stream.%20At%20every%20later%20slice%2C%20the%20arriving%20alignment%20measures%20whether%20the%20blocks%20put%20the%20axis%20back.%0A%0A%20%20%20%20%23%23%23%20Measurements%0A%0A%20%20%20%20One%20teacher-forced%20pass%20per%20run%20per%20intervention%2C%20over%20all%205%2C832%20lines.%20From%20each%20pass%3A%0A%0A%20%20%20%20-%20**The%20response**%20per%20line%3A%20the%20log-softmax%20over%20the%20vocabulary%20at%20the%20%60%3D%60%20position.%20From%20it%20we%20read%20the%20probability%20on%20the%20correct%20answer%2C%20the%20argmax%2C%20and%20the%20mass%20outside%20the%20color%20vocabulary.%0A%20%20%20%20-%20**The%20write**%20per%20(slice%2C%20line%2C%20position)%3A%20the%20angle%20between%20the%20arriving%20state%20and%20the%20edited%20state.%20The%20pipeline%20asserts%20that%20this%20equals%20%24%5Carcsin%7C%5Calpha_%7B%5Ctext%7Barriving%7D%7D%7C%24%20at%20every%20site%2C%20so%20where%20a%20map%20reads%20more%20easily%20we%20report%20the%20write%20as%20that%20alignment.%0A%20%20%20%20-%20**The%20clean%20alignment%20map**%20per%20run%2C%20from%20the%20un-intervened%20pass.%20This%20is%20the%20same%20array%20ex-2.1.10%20published%2C%20recomputed%20here%20so%20every%20quantity%20comes%20from%20one%20code%20path.%0A%20%20%20%20-%20**The%20final-state%20displacement**%20per%20line%3A%20the%20angle%20between%20the%20intervened%20and%20clean%20states%20at%20the%20last%20slice%2C%20%60%3D%60%20position.%20That%20is%20the%20state%20the%20logits%20are%20read%20from.%20We%20report%20it%20in%20radians%2C%20the%20same%20scale%20as%20a%20write.%0A%0A%20%20%20%20**The%20undesigned%20response**%2C%20on%20red%20lines%20under%20the%20primary%20intervention%2C%20without%20a%20gate.%20We%20record%20the%20mass%20outside%20the%20color%20vocabulary%3B%20how%20far%20the%20decoded%20answer%20sits%20from%20the%20true%20mix%2C%20in%20unit-cube%20units%3B%20which%20answer%20it%20is%20(the%20visible%20operand%2C%20the%20red%20operand%20itself%2C%20the%20true%20answer%2C%20a%20one-step%20neighbor%20of%20it%2C%20or%20something%20else)%3B%20and%20seed%20agreement%2C%20the%20fraction%20of%20red%20lines%20on%20which%20at%20least%20five%20of%20the%20nine%20seeds%20decode%20the%20same%20answer.%20These%20are%20the%20reference%20numbers%20for%20the%20fallback%20experiment.%0A%0A%20%20%20%20**Noise%20floor.**%20Clean%20accuracy%20on%20every%20group%20is%20at%20or%20near%201.0%20in%20every%20run%2C%20so%20the%20seed%20scatter%20that%20matters%20is%20in%20the%20intervened%20statistics.%20For%20each%20group%20statistic%20we%20compute%20the%20pooled%20between-seed%20standard%20deviation%20per%20condition%20before%20reading%20any%20comparison.%20A%20difference%20between%20arms%20or%20tiers%20smaller%20than%20twice%20that%20value%20is%20reported%20as%20not%20resolved.%20Gates%20score%20seed%20means%20against%20fixed%20thresholds%20and%20do%20not%20use%20the%20floor.%0A%0A%20%20%20%20%23%23%23%20Conditions%0A%0A%20%20%20%20Two%20conditions%20from%20ex-2.1.10%2C%20both%20trained%20on%20the%20same%20corpus%20with%20the%20same%20seeds%20and%20code%20path%3A%0A%0A%20%20%20%20-%20**anchored**%20(%60either-t100%60%2C%209%20seeds)%20%E2%80%94%20the%20D2.1%20recipe%3A%20pooled%20either-operand%20labels%2C%20the%20anchor%20at%20%CE%BB%20%3D%200.1%2C%20the%20anti-subspace%20term%20at%20the%20ex-2.1.8%20operating%20point.%20This%20is%20the%20primary%20condition%2C%20and%20the%20one%20every%20hypothesis%20scores.%0A%20%20%20%20-%20**un-anchored**%20(%60lam0%60%2C%203%20seeds)%20%E2%80%94%20the%20anchor%20weight%20at%20zero.%20Nothing%20was%20placed%20on%20e%E2%82%81%2C%20so%20projecting%20it%20out%20applies%20the%20operator%20to%20a%20model%20with%20nothing%20on%20the%20axis.%20That%20calibrates%20what%20a%20write%20of%20this%20kind%20costs%20when%20it%20removes%20no%20concept.%0A%0A%20%20%20%20**Arms**%2C%20on%20the%20anchored%20condition.%20Each%20changes%20one%20thing%20from%20the%20primary%20intervention%2C%20and%20we%20report%20the%20same%20statistics%20for%20each%20without%20gates%3A%0A%0A%20%20%20%20%7C%20arm%20%7C%20what%20changes%20%7C%20question%20%7C%0A%20%20%20%20%7C%20---%20%7C%20---%20%7C%20---%20%7C%0A%20%20%20%20%7C%20%60operands%60%20%7C%20positions%20restricted%20to%20op1%20and%20op2%20%7C%20does%20the%20edit%20need%20the%20syntax%20positions%2C%20whose%20embeddings%20carry%20a%20constant%20component%20on%20the%20axis%3F%20%7C%0A%20%20%20%20%7C%20%60embedding%60%20%7C%20slice%200%20only%20%7C%20is%20removing%20the%20token's%20component%20enough%2C%20or%20do%20the%20blocks%20re-derive%20red%3F%20%7C%0A%20%20%20%20%7C%20%60last%60%20%7C%20slice%204%20only%20%7C%20the%20one-readout%20case%2C%20closest%20to%20M1's%20geometry%20%7C%0A%20%20%20%20%7C%20%60lobe%60%20%7C%20the%20M1%20lobe%2C%20threshold%200.5%2C%20in%20place%20of%20the%20plain%20projection%20%7C%20does%20leaving%20the%20low-alignment%20bulk%20alone%20keep%20the%20removal%20and%20cut%20the%20collateral%3F%20%7C%0A%20%20%20%20%7C%20%60ablate%60%20%7C%20the%20weights%20that%20read%20and%20write%20e%E2%82%81%20zeroed%2C%20then%20re-normalized%20%7C%20the%20permanent%20removal%20from%20M1%2C%20scored%20under%20H5%20%7C%0A%0A%20%20%20%20**The%20post-hoc%20tier**%2C%20on%20the%20un-anchored%20condition%2C%20no%20gates.%20For%20each%20%60lam0%60%20run%20we%20fit%20one%20direction%20per%20slice%20on%20its%20own%20op1-position%20states%20(one%20line%20per%20color%2C%20216%20states)%2C%20then%20apply%20it%20at%20that%20slice%20at%20every%20position%20through%20the%20same%20scorer.%20Three%20fits%20beside%20e%E2%82%81%20itself%3A%20diff-in-means%20between%20the%20colors%20at%20or%20above%20the%20red%20dose%20and%20the%20rest%2C%20a%20ridge%20probe%20for%20redness%2C%20and%20LEACE%20for%20redness.%20This%20calibrates%20what%20a%20searched-for%20direction%20removes%20from%20a%20model%20that%20was%20never%20asked%20to%20place%20one.%0A%0A%20%20%20%20**The%20strength%20sweep**%2C%20on%20the%20anchored%20condition%3A%20the%20primary%20intervention%20at%20%CE%B3%20%E2%88%88%20%7B0.25%2C%200.5%2C%200.75%2C%201%7D%2C%20where%20%CE%B3%20is%20the%20fraction%20of%20the%20e%E2%82%81%20component%20removed.%20Exploratory%20(E1).%0A%20%20%20%20%22%22%22)%0A%20%20%20%20return%0A%0A%0A%40app.cell(hide_code%3DTrue)%0Adef%20_()%3A%0A%20%20%20%20mo.md(r%22%22%22%0A%20%20%20%20%23%23%20Hypotheses%0A%0A%20%20%20%20All%20on%20the%20anchored%20condition%20under%20the%20primary%20intervention%20unless%20stated%3B%20seed%20means%20over%20nine%20runs.%0A%0A%20%20%20%20-%20**H1.**%20Suppression%20bites.%20Seed-mean%20exact-match%20accuracy%20on%20red%20lines%20falls%20to%200.5%20or%20below%2C%20from%20a%20clean%20accuracy%20of%201.0.%20Partial%3A%20between%200.5%20and%200.8.%20Contrary%3A%20accuracy%20stays%20above%200.8%2C%20which%20would%20mean%20the%20model%20reads%20the%20color%20of%20the%20operand%20from%20somewhere%20the%20axis%20does%20not%20reach.%0A%20%20%20%20-%20**H2.**%20Suppression%20is%20selective.%20Seed-mean%20accuracy%20on%20non-red%20lines%20stays%20within%200.02%20of%20the%20clean%20accuracy.%20Partial%3A%20within%200.05.%20The%20gate%20is%20the%20width%20every%20D2.1%20task%20gate%20used.%0A%0A%20%20%20%20%20%20%20%20Projecting%20out%20an%20axis%20costs%20something%20even%20where%20nothing%20was%20placed%20on%20it.%20So%20we%20report%20the%20un-anchored%20row%20under%20the%20same%20operator%20beside%20this%20one%2C%20to%20say%20how%20much%20of%20any%20deficit%20comes%20from%20the%20operator%20rather%20than%20from%20the%20removal.%0A%20%20%20%20%20%20%20%20%3C!--%20REVIEW%3A%20H2%20was%20also%20stated%20as%20%22the%20behavioral%20half%20of%20the%20bound%20claim%22%2C%20with%20a%20miss%20under%20an%20intact%20H4%20read%20as%20%22a%20finding%20rather%20than%20a%20failure%20of%20the%20bound%22.%20That%20gave%20one%20bullet%20two%20verdicts.%20H2%20is%20now%20the%20task%20gate%20alone%2C%20with%20one%20verdict%3B%20what%20a%20miss%20means%20for%20the%20bound%20is%20interpretation%2C%20and%20lives%20in%20the%20H4%20analysis.%20Verify%3A%20the%20H4%20section%20says%20how%20an%20H2%20miss%20is%20read%2C%20and%20H2's%20verdict%20line%20in%20Findings%20is%20a%20plain%20pass%2Fpartial%2Ffail.%20--%3E%0A%20%20%20%20-%20**H3.**%20Damage%20grades%20with%20dose.%20We%20bin%20every%20line%20by%20dose%20at%20edges%200.2%2C%200.4%2C%200.6%2C%20and%200.8.%20Seed-mean%20damage%20is%20non-decreasing%20across%20the%20five%20bins%2C%20allowing%20a%20dip%20between%20adjacent%20bins%20of%20at%20most%200.02%2C%20and%20damage%20in%20the%20top%20bin%20exceeds%20the%20bottom%20bin%20by%20at%20least%200.5.%20Partial%3A%20one%20dip%20larger%20than%200.02%2C%20with%20the%20ends%20still%20ordered.%20We%20prescribe%20no%20shape%20between%20the%20ends.%20The%20D2.1%20alignment%20response%20was%20sigmoid%20in%20redness%20rather%20than%20linear%2C%20so%20the%20grading%20may%20be%20a%20step%20rather%20than%20the%20smooth%20%24%5Ccos%5E2%24%20curve%20M1%20saw.%0A%20%20%20%20-%20**H4.**%20The%20write%20on%20non-red%20lines%20stays%20within%20the%20bound%20the%20clean%20geometry%20sets%2C%20at%20every%20slice.%20At%20a%20given%20(slice%2C%20position)%20site%2C%20the%20bound%20is%20%24%5Carcsin%24%20of%20the%20seed-mean%2099th-percentile%20clean%20alignment%20over%20non-red%20lines%20there%2C%20taken%20from%20the%20un-intervened%20map.%20The%20measurement%20is%20the%20seed-mean%2099th-percentile%20write%20over%20the%20same%20lines%20under%20the%20primary%20intervention.%20At%20the%20embedding%20the%20two%20are%20equal%20by%20construction%2C%20and%20the%20pipeline%20asserts%20it.%20At%20each%20of%20the%20four%20deeper%20slices%2C%20and%20at%20every%20position%2C%20the%20measured%20write%20is%20at%20most%201.25%20times%20the%20bound.%20Partial%3A%20one%20site%20over%20by%20no%20more%20than%20a%20factor%20of%20two.%20Contrary%3A%20a%20block%20re-writes%20the%20axis%20on%20non-red%20lines%20after%20it%20was%20removed%20upstream.%20That%20would%20be%20a%20bypass%2C%20and%20the%20question%20of%20the%20layer%20sweep%20arriving%20early.%0A%20%20%20%20-%20**H5.**%20Permanent%20removal%20is%20selective%2C%20given%20the%20anti-subspace%20term.%20Under%20the%20%60ablate%60%20arm%2C%20the%20H1%20and%20H2%20gates%20both%20hold%3A%20red-line%20accuracy%20at%20or%20below%200.5%2C%20and%20non-red%20accuracy%20within%200.02%20of%20clean.%20Partial%3A%20one%20of%20the%20two%20holds.%20M1%20found%20that%20ablation%20was%20selective%20only%20when%20repulsion%20had%20cleared%20the%20axis%2C%20and%20the%20D2.1%20recipe%20has%20that%20term%2C%20so%20this%20is%20the%20transformer%20form%20of%20the%20headline%20result%20from%20M1.%0A%20%20%20%20%22%22%22)%0A%20%20%20%20return%0A%0A%0A%40app.cell(hide_code%3DTrue)%0Adef%20_()%3A%0A%20%20%20%20mo.md(r%22%22%22%0A%20%20%20%20%23%23%20Suppression%20bites%20(H1)%0A%0A%20%20%20%20%2F%2F%2F%20admonition%20%7C%20TODO%0A%20%20%20%20A%20table%20of%20clean%20and%20intervened%20accuracy%20under%20the%20primary%20intervention%2C%20seed%20mean%20with%20the%20seed%20range%2C%20for%20the%20red%20lines%2C%20the%20non-red%20lines%2C%20and%20all%20lines.%20The%20calibration%20row%20is%20the%20un-anchored%20condition%20under%20the%20same%20operator.%20Beside%20it%2C%20the%20per-line%20response%20on%20red%20lines%3A%20a%20strip%20or%20histogram%20of%20the%20intervened%20probability%20on%20the%20correct%20answer%2C%20per%20seed%2C%20so%20that%20a%20bimodal%20response%20(some%20lines%20fully%20removed%2C%20some%20intact)%20shows%20up%20as%20such%20instead%20of%20averaging%20into%20the%20gate.%0A%0A%20%20%20%20Expected%3A%20red-line%20accuracy%20far%20below%200.5%2C%20and%20the%20un-anchored%20row%20at%201.0.%20Contrary%3A%20red-line%20accuracy%20above%200.8%2C%20with%20the%20color%20of%20the%20operand%20still%20read%20from%20the%20off-axis%20remainder%20of%20its%20embedding.%20The%20%60leak_r2%60%20figure%20from%20ex-2.1.10%20says%20how%20much%20redness%20the%20off-axis%20state%20carries%20at%20op1%2C%20and%20that%20is%20the%20first%20number%20to%20look%20at.%0A%20%20%20%20%2F%2F%2F%0A%0A%20%20%20%20%23%23%20Selectivity%20(H2)%0A%0A%20%20%20%20%2F%2F%2F%20admonition%20%7C%20TODO%0A%20%20%20%20The%20non-red%20row%20of%20the%20H1%20table%2C%20scored%20against%20the%200.02%20gate%2C%20and%20the%20same%20row%20for%20every%20arm%20and%20for%20the%20post-hoc%20tier.%20One%20figure%20carries%20H2%20and%20H3%20together%3A%20damage%20against%20dose%2C%20per%20line%2C%20which%20is%20the%20grading%20cloud%20from%20D2.1%20with%20damage%20on%20the%20y%20axis.%20The%20non-red%20bulk%20should%20sit%20on%20the%20floor%20and%20the%20red%20lines%20near%20the%20ceiling.%0A%0A%20%20%20%20Expected%3A%20non-red%20accuracy%20within%20the%20gate%2C%20the%20%60operands%60%20arm%20no%20worse%2C%20the%20%60lobe%60%20arm%20no%20worse.%20Contrary%3A%20a%20non-red%20deficit%20larger%20than%200.02%2C%20read%20against%20H4%20as%20that%20section%20describes%3B%20or%20a%20deficit%20that%20the%20%60operands%60%20arm%20removes%2C%20which%20would%20place%20the%20collateral%20in%20the%20constant%20component%20of%20the%20syntax%20positions.%0A%20%20%20%20%2F%2F%2F%0A%0A%20%20%20%20%23%23%20Grading%20(H3)%0A%0A%20%20%20%20%2F%2F%2F%20admonition%20%7C%20TODO%0A%20%20%20%20Seed-mean%20damage%20per%20dose%20bin%2C%20with%20the%20seed%20range%2C%20as%20a%20step%20chart%20over%20the%20five%20bins%3B%20the%20same%20for%20the%20%60lobe%60%20arm%20and%20for%20the%20un-anchored%20condition.%20Expected%3A%20a%20monotone%20rise%2C%20possibly%20a%20step%20between%20the%200.6%20and%200.8%20edges%20rather%20than%20a%20ramp%2C%20and%20the%20un-anchored%20condition%20flat.%20Contrary%3A%20an%20inversion%2C%20with%20a%20middle%20bin%20damaged%20more%20than%20the%20bin%20above%20it.%20That%20would%20mean%20the%20axis%20carries%20something%20other%20than%20redness%20for%20those%20colors%2C%20and%20the%20per-color%20breakdown%20under%20E3%20says%20which%20colors.%0A%20%20%20%20%2F%2F%2F%0A%0A%20%20%20%20%23%23%20The%20write%20bound%20(H4)%0A%0A%20%20%20%20%2F%2F%2F%20admonition%20%7C%20TODO%0A%20%20%20%20Two%20maps%20of%20the%20non-red%2099th-percentile%20alignment%20per%20(slice%2C%20position)%3A%20the%20clean%20bound%2C%20and%20the%20arriving%20alignment%20under%20intervention%2C%20on%20the%20shared%200%E2%80%931%20scale%20the%20D2.1%20figures%20use.%20A%20table%20beside%20them%20gives%20the%20ratio%20at%20each%20site.%20The%20embedding%20row%20is%20the%20identity%20check.%0A%0A%20%20%20%20Expected%3A%20the%20deeper%20rows%20at%20or%20below%20the%20bound.%20The%20blocks%20were%20trained%20with%20the%20anti-subspace%20term%20on%20every%20state%2C%20so%20a%20non-red%20state%20arriving%20without%20its%20axis%20component%20is%20close%20to%20what%20they%20saw%20in%20training.%20Contrary%3A%20a%20site%20where%20the%20arriving%20alignment%20exceeds%20the%20bound%20by%20more%20than%20a%20quarter%2C%20most%20likely%20a%20syntax%20position%20at%20the%20second%20slice%2C%20where%20the%20smoke%20test%20saw%20one%20seed%20do%20exactly%20that.%20If%20that%20happens%2C%20the%20%60operands%60%20and%20%60lobe%60%20arms%20say%20whether%20it%20matters%20to%20behavior%2C%20and%20E5%20says%20whether%20the%20edit%20grows%20or%20shrinks%20on%20its%20way%20to%20the%20logits.%0A%0A%20%20%20%20H2%20is%20read%20beside%20this%20section.%20The%20bound%20covers%20the%20write%20and%20behavior%20is%20a%20prediction%20against%20it%2C%20so%20an%20H2%20miss%20with%20the%20bound%20intact%20is%20amplification%20by%20later%20blocks%3A%20a%20finding%20for%20the%20layer%20sweep%2C%20and%20H2%20fails%20as%20stated%20while%20H4%20stands.%20An%20H2%20miss%20with%20H4%20also%20missed%20is%20the%20bound%20failing%2C%20which%20is%20the%20plan's%20second%20risk.%0A%20%20%20%20%2F%2F%2F%0A%0A%20%20%20%20%23%23%20Permanent%20removal%20(H5)%0A%0A%20%20%20%20%2F%2F%2F%20admonition%20%7C%20TODO%0A%20%20%20%20The%20H1%20and%20H2%20rows%20for%20the%20%60ablate%60%20arm%2C%20beside%20the%20rows%20for%20the%20projection.%20Expected%3A%20the%20same%20removal%20and%20the%20same%20selectivity%2C%20since%20a%20stream%20that%20never%20carries%20e%E2%82%81%20is%20what%20the%20projection%20produces%20at%20every%20slice%2C%20up%20to%20the%20gain%20on%20the%20surviving%20components.%0A%0A%20%20%20%20Contrary%3A%20ablation%20biting%20less%20than%20projection%2C%20which%20would%20mean%20the%20blocks%20rebuild%20red%20from%20the%20off-axis%20embedding%20when%20the%20axis%20is%20unavailable%20from%20the%20start%3B%20or%20ablation%20costing%20more%20on%20non-red%20lines%2C%20which%20would%20mean%20the%20read%20and%20write%20matrices%20use%20the%20axis%20for%20something%20the%20anti-subspace%20term%20did%20not%20clear.%0A%20%20%20%20%2F%2F%2F%0A%0A%20%20%20%20%23%23%20The%20undesigned%20response%0A%0A%20%20%20%20%2F%2F%2F%20admonition%20%7C%20TODO%0A%20%20%20%20No%20gate.%20For%20red%20lines%20under%20the%20primary%20intervention%3A%20the%20mass%20outside%20the%20color%20vocabulary%3B%20how%20far%20the%20decoded%20answer%20sits%20from%20the%20true%20mix%3B%20the%20composition%20of%20decoded%20answers%20(visible%20operand%2C%20red%20operand%2C%20true%20answer%2C%20one-step%20neighbor%2C%20other)%2C%20as%20a%20stacked%20bar%20per%20seed%3B%20and%20the%20seed-agreement%20fraction.%0A%0A%20%20%20%20Expected%3A%20completions%20stay%20in%20the%20color%20vocabulary%2C%20the%20decoded%20answer%20sits%20nearer%20the%20visible%20operand%20than%20the%20true%20mix%2C%20and%20agreement%20across%20seeds%20is%20low.%20An%20untrained%20response%20should%20scatter%20by%20seed.%0A%20%20%20%20%2F%2F%2F%0A%0A%20%20%20%20%23%23%20The%20post-hoc%20tier%0A%0A%20%20%20%20%2F%2F%2F%20admonition%20%7C%20TODO%0A%20%20%20%20No%20gate.%20The%20H1%E2%80%93H3%20statistics%20for%20each%20fitted%20direction%20on%20the%20un-anchored%20condition%2C%20beside%20e%E2%82%81%20on%20the%20same%20condition%20and%20beside%20the%20anchored%20primary.%20The%20comparison%20to%20read%3A%20how%20much%20red%20damage%20a%20searched-for%20direction%20buys%20per%20unit%20of%20non-red%20damage%2C%20compared%20with%20the%20placed%20axis.%0A%0A%20%20%20%20Expected%3A%20diff-in-means%20and%20the%20probe%20remove%20some%20red-line%20accuracy%20at%20a%20real%20non-red%20cost%2C%20LEACE%20less%20of%20both%2C%20and%20none%20of%20them%20come%20close%20to%20the%20selectivity%20of%20the%20placed%20axis.%20A%20post-hoc%20direction%20that%20matched%20the%20anchored%20result%20would%20mean%20the%20un-anchored%20model%20already%20keeps%20redness%20on%20a%20single%20linear%20direction%20at%20op1%2C%20which%20the%20probe%20maps%20in%20ex-2.1.12%20make%20plausible%20at%20the%20embedding.%20The%20per-slice%20breakdown%20says%20where%20the%20two%20part%20company.%0A%20%20%20%20%2F%2F%2F%0A%0A%20%20%20%20%23%23%20Exploratory%20analyses%0A%0A%20%20%20%20%2F%2F%2F%20admonition%20%7C%20TODO%0A%20%20%20%20Preregistered%20as%20exploratory%2C%20no%20gates.%0A%0A%20%20%20%20**E1%20%E2%80%94%20strength.**%20Red-line%20and%20non-red-line%20accuracy%20and%20damage%20at%20%CE%B3%20%E2%88%88%20%7B0.25%2C%200.5%2C%200.75%2C%201%7D.%20The%20write%20at%20strength%20%CE%B3%20is%20closed-form%20in%20%CE%B1%2C%20so%20the%20x%20axis%20can%20be%20drawn%20either%20as%20%CE%B3%20or%20as%20the%20write%20on%20red%20operands%3B%20the%20question%20is%20whether%20the%20response%20is%20proportional%20to%20the%20write%20or%20thresholded.%0A%0A%20%20%20%20**E2%20%E2%80%94%20where%20the%20edit%20acts.**%20The%20H1%E2%80%93H3%20statistics%20for%20the%20%60embedding%60%20and%20%60last%60%20arms%20against%20the%20primary.%20If%20%60embedding%60%20alone%20bites%2C%20the%20concept%20is%20read%20off%20the%20token%20and%20the%20blocks%20do%20not%20re-derive%20it.%20If%20only%20the%20primary%20intervention%2C%20at%20every%20slice%2C%20bites%2C%20the%20bypass%20test%20the%20D1.3%20post%20left%20open%20gets%20its%20first%20transformer%20data%20point.%0A%0A%20%20%20%20**E3%20%E2%80%94%20damage%20against%20alignment.**%20Per%20line%2C%20damage%20against%20the%20clean%20alignment%20at%20the%20position%20of%20the%20red%20operand%2C%20at%20each%20slice.%20In%20M1%20the%20relation%20was%20error%20%E2%88%9D%20%24%5Ccos%5E2%24.%20Also%20per%20color%3A%20the%20mean%20damage%20of%20lines%20with%20that%20color%20as%20an%20operand%2C%20against%20the%20alignment%20of%20that%20color.%20This%20is%20the%20per-color%20breakdown%20the%20contrary%20case%20of%20H3%20asks%20for.%0A%0A%20%20%20%20**E4%20%E2%80%94%20the%20lobe.**%20The%20H1%E2%80%93H4%20statistics%20for%20the%20%60lobe%60%20arm.%20Its%20threshold%20of%200.5%20sits%20above%20the%20constant%20component%20of%20the%20syntax%20embeddings%20and%20below%20the%20alignment%20of%20the%20red%20operand%2C%20so%20by%20design%20it%20should%20remove%20red%20and%20leave%20the%20syntax%20states%20alone.%20Its%20write%20map%20is%20the%20H4%20figure%20with%20the%20syntax%20columns%20emptied.%0A%0A%20%20%20%20**E5%20%E2%80%94%20amplification.**%20Per%20non-red%20line%2C%20the%20final-state%20displacement%20at%20the%20%60%3D%60%20position%20against%20the%20sum%20of%20the%20writes%20on%20that%20line%20at%20the%20prompt%20positions%2C%20over%20all%20slices.%20Both%20are%20angles%2C%20so%20their%20ratio%20is%20the%20gain%20the%20blocks%20apply%20to%20the%20edit%20on%20its%20way%20to%20the%20logits%3B%20a%20ratio%20above%20one%20is%20amplification.%20We%20read%20it%20for%20the%20primary%20intervention%2C%20and%20for%20the%20%60last%60%20arm%2C%20where%20no%20block%20sits%20after%20the%20edit.%20The%20%60last%60%20arm%20is%20the%20no-amplification%20reference.%0A%20%20%20%20%3C!--%20REVIEW%3A%20the%20displacement%20is%20now%20an%20angle%20rather%20than%20a%20distance%2C%20so%20the%20ratio%20is%20dimensionless%3B%20and%20the%20%60last%60%20arm's%20ratio%20is%20not%20one%20by%20construction%2C%20since%20the%20denominator%20sums%20the%20writes%20at%20all%20four%20prompt%20positions%20while%20the%20numerator%20is%20the%20displacement%20at%20%60%3D%60%20alone.%20Verify%3A%20under%20%60last%60%2C%20the%20numerator%20equals%20the%20write%20at%20(slice%204%2C%20%60%3D%60)%2C%20so%20the%20ratio%20is%20one%20only%20when%20the%20other%20three%20prompt%20writes%20vanish.%20--%3E%0A%0A%20%20%20%20%2F%2F%2F%0A%0A%20%20%20%20%23%23%20Discussion%0A%0A%20%20%20%20%2F%2F%2F%20admonition%20%7C%20TODO%0A%20%20%20%20Interpretation%20only%2C%20after%20the%20results.%20What%20the%20removal%20and%20its%20selectivity%20let%20the%20D2.2%20plan%20say%20about%20the%20anchored-op%20experiments%20that%20follow%3B%20which%20operator%20the%20damage%20figures%20pick%20for%20them%2C%20which%20the%20%5Bshaped-suppression%20item%5D(%2Ftodo%2Fscience%2Fshaped-suppression-rather-than-projecting-whole-axis.md)%20is%20waiting%20on%3B%20what%20the%20undesigned%20response%20says%20the%20fallback%20experiment%20must%20improve%20on%3B%20and%20what%20the%20write%20bound%20and%20E5%20say%20about%20the%20layer%20sweep.%20No%20re-derivation%20of%20the%20findings.%0A%20%20%20%20%2F%2F%2F%0A%20%20%20%20%22%22%22)%0A%20%20%20%20return%0A%0A%0Aif%20__name__%20%3D%3D%20%22__main__%22%3A%0A%20%20%20%20app.run()%0A
a0dac708eba38cb4c26a3da18ff56c25