Thursday, December 1, 2022 - 10:00

Addressing knowledge-rich tasks in the real world

Speaker: Eunsol Choi
Location: Allen Center 305 and Zoom https://ucsc.zoom.us/j/92375270528?pwd=M0NGY0dVWEZKT3JlWHpVUGlIWG83Zz09
Abstract: To address knowledge-rich tasks such as question answering and fact checking, NLP models should combine knowledge from multiple sources – memorized knowledge in the language model and passages retrieved from an evidence corpus. Contrary to this, prior work has made simplifying assumptions that knowledge sources are consistent with each other, up-to-date, and available. In this talk, I will discuss challenges and opportunities for building a NLP model in a real world that is open-ended and constantly changing. I will first describe how existing models behave under different types of knowledge conflicts. Then, I will propose paths forward for handling knowledge conflicts -- teaching models to detect conflicting information and generating paragraph-level answers that can elaborate multiple viewpoints.

Bio: Eunsol Choi is an assistant professor in the Computer Science department at the University of Texas at Austin and a visiting researcher at Google AI. Her research area spans natural language processing and machine learning. She is particularly interested in interpreting and reasoning about text in a rich real world context. She received a Ph.D. from University of Washington and B.A from Cornell University. She is a recipient of Facebook research fellowship, Google faculty research award, and outstanding paper award at EMNLP 2021.

Tuesday, November 15, 2022 - 10:00

Interaction and Natural Language Learning

Speaker: Yoav Artzi, Cornell Tech
Location: Allen Center 305 and Zoom (Link: https://washington.zoom.us/j/99598045832 [washington.zoom.us])
Abstract: This talk focuses on the challenges and opportunities that interactions provide for natural language learning. I will show how collaborative interactions enable continual learning, where agentive systems improve over time through interaction. I will describe a game-like environment that instantiates collaborative interactions with natural language coordination, and show how it creates a contextual bandit learning scenario for language production (i.e., generation). I will also discuss language-conditioned reinforcement learning (RL), a research area that remains challenging to pursue despite its promise for both research and application. A key challenge hindering progress is computing rewards that require resolving language semantics. I will describe a language-conditioned RL benchmark using a new approach for reward computation, striking a balance between research accessibility and retaining the complexities and nuances of natural language.

Bio: Yoav Artzi is an Associate Professor in the Department of Computer Science and Cornell Tech at Cornell University. His research focuses on developing learning methods for natural language understanding and generation in automated interactive systems. He received an NSF CAREER award, and his work was acknowledged by awards and honorable mentions at ACL, EMNLP, NAACL, and IROS. Yoav holds a B.Sc. from Tel Aviv University and a Ph.D. from the University of Washington.

Thursday, May 14, 2020 - 11:00

TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages

Speaker: Jonathan Clark, Google AI
Location: Virtual
Abstract: Confidently making progress on multilingual modeling requires challenging, trustworthy evaluations. We present TyDi QA —a question answering dataset covering 11 typologically diverse languages with 204K question-answer pairs. The languages of TyDi QA are diverse with regard to their typology—the set of linguistic features each language expresses—such that we expect models performing well on this set to generalize across a large number of the world’s languages.

We present a quantitative analysis of the data quality and example-level qualitative linguistic analyses of observed language phenomena that would not be found in English-only corpora. To provide a realistic information-seeking task and avoid priming effects, questions are written by people who want to know the answer, but don’t know the answer yet, and the data is collected directly in each language without the use of translation.

In this talk, we’ll argue for the immediate practical value of building QA systems that work for people that don’t speak English, the modeling challenge of coping with lower-resource languages, and the scientific value of learning what it takes to model the variety presented by human languages.

Bio: Jonathan Clark is a research scientist at Google Research in Seattle. His research goal is to build NLP systems that are helpful to people regardless of what language they speak. Previously, he was a member of the machine translation team at Microsoft Research, helping to build Skype Translator and Microsoft Custom Translator. He holds a Ph.D. from Carnegie Mellon University.

GitHub (baselines & eval code): https://github.com/google-research-datasets/tydiqa
Website & glossed examples: https://ai.google.com/research/tydiqa/
Paper: https://arxiv.org/abs/2003.05002

Thursday, December 12, 2019 - 12:00

How will we know when machines can read?

Speaker: Matt Gardner, AI2 Irvine
Location: CSE 305
Abstract: The task of machine reading comprehension, asking a machine questions about a passage of text to probe its understanding, has seen a dramatic surge in popularity in recent years. According to some metrics, we now have machines that perform as well as humans on this task. Yet no serious researcher actually believes that machines can read, despite their performance on some reading comprehension benchmarks. What would it take to convince ourselves that a machine understood a passage of text? Can we devise a benchmark that would let us measure progress towards that goal? In this talk I try to outline what such a benchmark might look like, and share some initial progress towards building one.

Bio: Matt is a senior research scientist at the Allen Institute for AI on the AllenNLP team. His research focuses primarily on getting computers to read and answer questions, dealing both with open domain reading comprehension and with understanding question semantics in terms of some formal grounding (semantic parsing). He is particularly interested in cases where these two problems intersect, doing some kind of reasoning over open domain text. He is the original architect of the AllenNLP toolkit, and he co-hosts the NLP Highlights podcast with Waleed Ammar and Pradeep Dasigi.

Tuesday, November 19, 2019 - 10:30

Evaluating Recent Progress Toward General-Purpose Language Understanding Models

Speaker: Sam Bowman
Location: CSE 305

Abstract: The GLUE and SuperGLUE shared-task benchmarks aim to measure progress toward the goal of building general-purpose pretrained neural network models for language understanding. This goal turns out to have been widely shared, and these benchmarks have become a significant target for research in the NLP and machine learning communities. In this talk, I'll review the motivations behind these benchmarks and what these benchmarks can tell us about recent progress in NLP, and raise a few (open!) questions about how we should measure further progress in this area.

Bio: Sam Bowman has been an assistant professor at NYU since 2016, when he completed PhD with Chris Manning and Chris Potts at Stanford. At NYU, Sam is jointly appointed between the new school-level Center for Data Science, which focuses on machine learning, and the Department of Linguistics. Sam's research focuses on data, evaluation techniques, and modeling techniques for sentence and paragraph understanding in natural language processing, and on applications of machine learning to scientific questions in linguistic syntax and semantics. Sam organized a twenty-three person research team at JSALT 2018 and received a 2015 EMNLP Best Resource Paper Award, a 2017 Google Faculty Research Award, and a 2019 *SEM Best Paper Award.

Thursday, October 31, 2019 - 10:30

Divergences in Neural Machine Translation

Speaker: Marine Carpuat, University of Maryland
Location: CSE 305
Abstract: While deep neural network models have dramatically improved the quality of machine translation (MT), truly breaking language barriers requires not only translating accurately, but also comparing what is said and how it is said across languages. In this talk, I will argue that modeling divergences from common assumptions about the data used to model machine translation (MT) can not only improve MT, but also help broaden the framing of MT to make it more responsive to user needs. I will first discuss recent work on automatically detecting cross-lingual semantic divergences, which occur when translation does not preserve meaning [Vyas & Carpuat, EMNLP 2019]. Next, I will introduce a training objective for neural sequence-to-sequence models that accounts for divergences between MT model hypotheses and reference human translation [Xu, Niu & Carpuat, NAACL 2019]. Finally, I will argue that translation does not necessarily need to preserve all properties of the input and introduce a family of models that let us tailor translation style while preserving input meaning [Niu, Rao & Carpuat, COLING 2017; Agrawal & Carpuat, EMNLP 2019].

Bio: Marine Carpuat is an Assistant Professor in Computer Science at the University of Maryland. Her research focuses on multilingual natural language processing and machine translation. Before joining the faculty at Maryland, Marine was a Research Scientist at the National Research Council Canada. She received a PhD in Computer Science and a MPhil in Electrical Engineering from the Hong Kong University of Science & Technology, and a Diplome d'Ingenieur from the French Grande Ecole Supelec. Marine is the recipient of an NSF CAREER award, research awards from Google and Amazon, best paper awards at *SEM and TALN, and an Outstanding Teaching Award.

Tuesday, October 22, 2019 - 10:30

Robot Control and Collaboration in Situated Instruction Following

Speaker: Yoav Artzi
Location: CSE 305

Abstract: I will present two projects studying the problem of learning to follow natural language instructions. I will present new datasets, a class of interpretable models for instruction following, learning methods that combine the benefits of supervised and reinforcement learning, and new evaluation protocols. In the first part, I will discuss the task of executing natural language instructions with a robotic agent. In contrast to existing work, we do not engineer formal representations of language meaning or the robot environment. Instead, we learn to directly map raw observations and language to low-level continuous control of a quadcopter drone. In the second part, I will propose the task of learning to follow sequences of instructions in a collaborative scenario, where both the user and the system execute actions in the environment and the user controls the system using natural language. To study this problem, we build CerealBar, a multi-player 3D game where a leader instructs a follower, and both act in the environment together to accomplish complex goals.
The two projects were led by Valts Blukis, Alane Suhr, and collaborators. Additional information about both projects is available here:
https://github.com/lil-lab/drif
http://lil.nlp.cornell.edu/cerealbar/

Bio: Yoav Artzi is an Assistant Professor in the Department of Computer Science and Cornell Tech at Cornell University. His research focuses on learning expressive models for natural language understanding, most recently in situated interactive scenarios. He received an NSF CAREER award, paper awards in EMNLP 2015, ACL 2017, and NAACL 2018, a Google Focused Research Award, and faculty awards from Google, Facebook, and Workday. Yoav holds a B.Sc. summa cum laude from Tel Aviv University and a Ph.D. from the University of Washington.

Tuesday, October 15, 2019 - 10:30

The Quest for Automated Story Generation

Speaker: Mark Riedl
Location: CSE 305

Abstract: Storytelling is a pervasive part of the human experience--we as humans tell stories to communicate, inform, entertain, and educate. In this talk, I will lay out the case for the study of storytelling through the lens of artificial intelligence. I will explore the grand challenge of building an intelligent systems that learn to tell stories. I will specifically look at two challenges pertaining to neural generative processes: control of text generation to achieve goal-driven behavior, and the challenge of maintaining and transferring long-term world knowledge. While both of these challenges are necessary for automated story generation, they also appear in other artificial intelligence and natural language processing tasks as well.

Bio: Dr. Mark Riedl is an Associate Professor in the Georgia Tech School of Interactive Computing and director of the Entertainment Intelligence Lab. Dr. Riedl’s research focuses on human-centered artificial intelligence—the development of artificial intelligence and machine learning technologies that understand and interact with human users in more natural ways. Dr. Riedl’s recent work has focused on story understanding and generation, computational creativity, explainable AI, and teaching virtual agents to behave safely. His research is supported by the NSF, DARPA, ONR, the U.S. Army, U.S. Health and Human Services, Disney, and Google. He is the recipient of a DARPA Young Faculty Award and an NSF CAREER Award.

Wednesday, October 9, 2019 - 12:45

The Ins and Outs of Preposition Semantics: Challenges in Comprehensive Corpus Annotation and Automatic Disambiguation

Speaker: Nathan Schneider
Location: CSE 305

Abstract: In most linguistic meaning representations that are used in NLP, prepositions fly under the radar. I will argue that they should instead be put front and center given their crucial status as linkers of meaning—whether for spatial and temporal relations, for predicate-driven roles, or in special constructions. To that end, we have sought to characterize and disambiguate semantic functions expressed by prepositions and possessives in English (Schneider et al., ACL 2018; https://github.com/nertnlp/streusle/), and similar markers in other languages (ongoing work on Korean, Hebrew, German, and Mandarin Chinese). This approach can be broadened to provide language-neutral, lexicon-free semantic relations in structured meaning representation parsing (Prange et al., CoNLL 2019; Shalev et al., DMR 2019).

Bio: Nathan Schneider is an annotation schemer and computational modeler for natural language. As Assistant Professor of Linguistics and Computer Science at Georgetown University, he looks for synergies between practical language technologies and the scientific study of language. He specializes in broad-coverage semantic analysis: designing linguistic meaning representations, annotating them in corpora, and automating them with statistical natural language processing techniques. A central focus in this research is the nexus between grammar and lexicon as manifested in multiword expressions and adpositions/case markers. He has inhabited UC Berkeley (BA in Computer Science and Linguistics), Carnegie Mellon University (Ph.D. in Language Technologies), and the University of Edinburgh (postdoc). Now a Hoya and leader of NERT, he continues to play with data and algorithms for linguistic meaning.

Wednesday, October 9, 2019 - 12:00

What Logic can Teach Neural Networks: Case Studies in Language

Speaker: Vivek Srikumar
Location: CSE 305

Abstract: Today, the most common approach for training neural networks involves minimizing task loss on large datasets. While this agenda has been undeniably successful, we may not have the luxury of annotated data every task or domain. Reducing dependence on labeled examples may require us to rethink how we supervise models. In this talk, I will describe recent work where we use knowledge to inform neural networks without introducing additional parameters. Declarative rules in logic can be systematically compiled into computation graphs that augment the structure of neural models, and also into regularizers that can use labeled or unlabeled examples. I will present experiments on text understanding tasks, which show that such declaratively constrained neural networks are not only more accurate, but also more consistent in their predictions across examples. This is joint work with my students Tao Li, Vivek Gupta and Maitrey Mehta.

Bio: Vivek Srikumar is an assistant professor in the School of Computing at the University of Utah. His research lies in the areas of natural learning processing and machine learning and has primarily been driven by questions arising from the need to reason about textual data with limited explicit supervision and to scale NLP to large problems. His work has been published in various AI, NLP and machine learning venues and received the best paper award at EMNLP 2014. His work has been supported by NSF and BSF, and gifts from Google, Nvidia and Intel. He obtained his Ph.D. from the University of Illinois at Urbana-Champaign in 2013 and was a post-doctoral scholar at Stanford University.