Researchers from Prefrontal AI and ETH Zurich introduced Latent Sparse Attention for ultra-long context models. The technique groups tokens into semantic clusters, achieving up to a 72-fold layer speedup at ten million tokens while matching full attention accuracy.
FireBrief
FireBrief