NLP Research Lab
Various Presenters (Allen School)
Research Talk
Thursday, January 11, 2024, 3:30 pm
Abstract
Speaker: Sewon Min
Rethinking Data Use in Language Modeling
Summary: Current language models rely on remembering information from static training data, which is fundamentally limited and prevents staying up-to-date. In this talk, I will motivate a new approach that uses data together with model parameters at inference time, addressing issues current models have.
Speaker: Margaret Li
Branch-Train-Merge: Efficient LLM Training
Summary: The prevailing paradigm for LLM training becomes ever more prohibitively expensive. Here, we will discuss LLM training approaches that leverage adaptive computation and sparsity, thus increasing training efficiency and enabling model customization during and after training.
Speaker: Orevaoghene Ahia
Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models
Summary: Subword tokenization has become the defacto method to segment text in the field of NLP. In this talk, we shed light on the current flaws of subword tokenization methods and their effects on model utility , inference and API costs in the current age of Commercial LLMs.
Speaker: Shangbin Feng
From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models
Summary: In this talk, we demonstrate the political bias propagation pipeline in language model training and applications: from opinions and perspectives in pretraining data, to inherent political biases of language models, to unfairness in downstream task applications.
Speaker: Niloofar Mireshghallah
Can LLMs Keep a Secret? Testing Privacy Implications of Language Models in Interactive Settings
Summary: In this talk, we draw attention to a new set of inference-time privacy risks of using LLMs in interactive settings, where they are fed different information from multiple sources and expected to reason about what to share in their outputs.