Are learning-curve shapes exactly two kinds — spectral readouts and competitions between coexisting solutions?

Status: open · laid 2026-09-27 · type: structural

Statement. The classic shapes a training curve takes appear to fall into two kinds, and the proposal is that the split is real — a difference in mechanism, not in appearance — and that asking whether it is exhaustive is the useful question.

Kind one: the curve reads out a spectrum. Something about the curve's geometry is in correspondence with the eigenvalues or singular values of a fixed operator, so measuring the curve measures the spectrum.

• Staircase plateaus in a deep linear network: one step per singular mode, plateau length a function of that mode's singular value. The curve's timing carries the spectrum (P-027).
• The double-descent peak at p = n: the risk spike is the smallest eigenvalue of the feature matrix collapsing. The curve's location and height carry a spectral edge (P-029).
• Scaling-law exponents: the power with which loss falls tracks the decay rate of the data or kernel eigenvalue spectrum. The curve's slope carries the decay.

Kind two: the curve records a competition. Two solutions coexist for the same task, and the curve is the record of one overtaking the other; nothing about its shape is a spectrum.

• Grokking: memorisation wins early and generalisation wins late, with the crossover set by regularisation rather than by any eigenvalue — which is why weight decay moves the snap.
• Sudden in-context-learning jumps: an in-weights solution is displaced by an in-context one, with the jump marking the crossing rather than the arrival of anything new.

The two questions, in order. Is the split real? — i.e. is there a mechanical test that sorts a given curve into a kind, rather than a family resemblance that breaks under examination. The candidate test is the one the two kinds differ on: a readout should move with the spectrum under an intervention that changes the spectrum and nothing else, and a competition should move with the relative strength of two solutions under an intervention that changes regularisation and nothing else. And is it exhaustive? — a third kind would be the interesting outcome, and the obvious candidate is a curve shaped by the optimiser's own instability rather than by either the spectrum or a competition (P-028's edge of stability is a curve of that sort, and it is not obvious which kind it belongs to, which is the probe's own best counterexample).

Why this is filed as structural and not as a lead. It proposes a coordinate — which kind of mechanism a curve shape belongs to — rather than a computation to run, and a coordinate earns its place in this project only by predicting an absence that is then found (docs/CHARTER.md, the failure mode; P-024 is the precedent, still held for want of exactly that). The absences this one would have to predict are unnamed so far, which is precisely why it is a probe: it is a candidate coordinate with its promotion bar not met and stated as unmet.

No prediction and no falsifier are filed here — both are the human's.

Cross-refs. P-027 (staircase), P-028 (edge of stability, the awkward case), P-029 (double descent) are the three instances, filed the same day this thread opened; lab/runs/037-grokking-multiplicative-characters.md is a grokking curve measured, hence an instance of kind two whose readout was taken from the weights rather than from the curve — which is itself a small piece of evidence that the two kinds want different instruments.

Back-links (2026-09-29)

Dated pointers added by the codifier from lab/backlinks.py; nothing above this heading was changed.

• Run 037 — confirmed (2026-09-27, measured). lab/runs/037-grokking-multiplicative-characters.md: CONFIRMED 2026-09-28 by the repo owner, scored on the prediction's intent.