Abstracts: NeurIPS 2024 with Weizhu Chen
Microsoft Research Podcast - Ein Podcast von Researchers across the Microsoft research community
Kategorien:
Next-token prediction trains a language model on all tokens in a sequence. VP Weizhu Chen discusses his team’s 2024 NeurIPS paper on how distinguishing between useful and “noisy” tokens in pretraining can improve token efficiency and model performance.Read the paperGet the code
