Researchers introduced TokenGate, a lightweight module that teaches language models to skim long documents during initial processing. The system prunes unimportant tokens early, accelerating time-to-first-token by up to 11.3 times without reducing task accuracy.
FireBrief
FireBrief