Researcher Jon Barron identified two numerical bugs in standard softmax and cross-entropy implementations across libraries like PyTorch and JAX. His simple code fix eliminates gradient errors, removes slingshot spikes, and accelerates model grokking.
FireBrief
FireBrief