Talks

Abstract: Despite large advances in neural text generation in terms of fluency, existing generation techniques are prone to hallucination and often produce output that is factually incorrect or structurally incoherent. In this talk, we study this problem from the perspectives of evaluation, modeling, and robustness. We first discuss how existing evaluation metrics like BLEU or ROUGE show poor correlation to human judgement when the reference text diverges from information in the source, a common phenomena in generation datasets. We propose a new metric, PARENT, which aligns n-grams to the source before computing precision and recall, making it considerably more robust to divergence. Next, we discuss modeling, proposing an exemplar-based approach to conditional text generation that aims to leverage training instances to build instance-specific decoders that can more easily capture style and structure. Results on 3 datasets show that our model achieves strong performance and outperforms comparable baselines. Lastly, we discuss generalization of neural generation in non-iid settings, focusing on the problem of zero shot translation — a challenging setup that tests models on translation directions they have not been optimized for at training time. We define the notion of zero-shot consistency and introduce a consistent agreement-based training method that results in a 2-3 BLEU zero shot improvement over strong baselines.

Collaborators: This is joint work with Maruan Al-Shedivat, Bhuwan Dhingra, Hao Peng, Manaal Faruqui, Ming-Wei Chang, William Cohen, and Dipanjan Das.

Bio: Ankur Parikh is a Senior Research Scientist at Google NYC and adjunct assistant professor at NYU. His primary interests are in natural language processing and machine learning. Ankur received his PhD from Carnegie Mellon in 2015 and his B.S.E. from Princeton University in 2009. He has received a best paper runner up award at EMNLP 2014 and a best paper in translational bioinformatics at ISMB 2011.

Abstract:
Psycholinguistics and computational linguistics are the two fields most dedicated to accounting for the computational operations required to understand natural language. Today, both fields find themselves responsible for understanding the behaviors and inductive biases of "black-box" systems: the human mind and artificial neural network (ANN) models, respectively. In this talk I highlight how the two fields can productively contribute to one another, with a focus on the study of syntactic processing. I first describe the surprisal theory of interpretation, prediction, and differential processing difficulty in human language comprehension. I then show how surprisal theory and controlled experimental paradigms from psycholinguistics can help us probe ANN language model behavior for evidence of human-like grammatical generalizations. We find that ANNs exhibit a range of subtle behaviors, including embedding-depth tracking and garden-pathing over long stretches of text, that suggest representations homologous to incremental syntactic state in human language processing. These ANNs also learn abstract word-order preferences and many generalizations about the long-distance filler-gap dependencies that are a hallmark of natural language syntax, perhaps most surprisingly including many filler-gap "island" constraints. However, even when trained on a human lifetime's worth of linguistic input these ANNs fail to learn a number of key basic facts about other core grammatical dependencies. Finally, I comment on the respects in which the departures of recurrent neural network language models from the predictions of the "competence" grammars developed in generative linguistics might provide a "performance" account of human language processing -- and on the respects in which they might not.

Bio:
Roger Levy is a professor in the Department of Brain and Cognitive Sciences at the Massachusetts Institute of Technology. He directs MIT's Computational Psycholinguistics Laboratory. Before joining MIT, he was faculty in the Department of Linguistics at UC San Diego and received his PhD in Linguistics from Stanford University. His research focuses on theoretical and applied questions in the processing and acquisition of natural language.

Abstract: In this talk, I will present a general framework for sequence modeling by exploring the segmental structures of the sequences. We first observe that segmental structure is a common pattern in many types of sequences, e.g., phrases in human languages. We then design a probabilistic model that is able to consider all valid segmentations for a sequence. We describe an efficient and exact dynamic programming algorithm for forward and backward computations. Due to the generality, it can be used as a loss function in many sequence tasks. We demonstrate our approach on text segmentation, speech recognition, machine translation and dialog policy learning. In addition to quantitative results, we also show that our approach can discover meaningful segments in their respective application contexts. (This is a joint work with many of my previous and current collaborators.)

Bio: Chong Wang is a research scientist at Google. Before Google, He worked at Microsoft Research and Baidu Silicon Valley AI Lab. He received his PhD from Princeton University. His research interests include machine learning and their applications to speech, translation and natural language understanding. He has won several best paper awards in top machine learning conferences and some of his work went into widely used products to serve the users from the globe. His homepage is https://chongw.github.io

Abstract:

    The big data boom in recent years covers a wide spectrum of heterogeneous data types, from text to image, video, speech, and multimedia. Most of the valuable information in such "big data" is encoded in natural language, which makes it accessible to some people—for example, those who can read that particular language—but much less amenable to computer processing beyond a simple keyword search.

    My focused research area, cross-source Information Extraction (IE) on a massive scale, aims to create the next generation of information access in which humans can communicate with computers in any natural language beyond keyword search, and computers can discover accurate, concise, and trustable information embedded in big data from heterogeneous sources.

    The goal of Information Extraction (IE) is to extract structured facts from a wide spectrum of heterogeneous unstructured data types. Traditional IE techniques are limited to a certain source X (X = a particular language, domain, limited number of pre-defined fact types, single data modality, ...). When moving from X to a new source Y, we need to start from scratch again by annotating a substantial amount of training data and developing Y-specific extraction capabilities.

    In this talk, I will present a new Universal IE paradigm to combine the merits of traditional IE (high quality and fine granularity) and Open IE (high scalability). This framework is able to discover schemas and extract facts from any input data in any domain, without any annotated training data, by integrating distributional semantics and symbolic semantics. It can also be extended to thousands of languages, thousands of fact types and multiple data modalities (text, images, videos) by constructing a multi-lingual multi-media multi-task common semantic space and then performing zero-shot transfer learning across sources. The resulting system was selected for DARPA 60 Anniversary.

Bio:

Heng Ji is the Edward P. Hamilton Chair Professor in Computer Science at Rensselaer Polytechnic Institute. She received her Ph.D. in Computer Science from New York University. Her research interests focus on Natural Language Processing, especially on Information Extraction and Knowledge Base Population. She was selected as "Young Scientist" and a member of the Global Future Council on the Future of Computing by the World Economic Forum in 2016, 2017 and 2018. She received "AI's 10 to Watch" Award by IEEE Intelligent Systems in 2013, NSF CAREER award in 2009, Google Research Awards in 2009 and 2014, IBM Watson Faculty Award in 2012 and 2014, and Bosch Research Awards in 2015, 2016 and 2017. She coordinated the NIST TAC Knowledge Base Population task since 2010, and led various government sponsored research projects, including the DARPA DEFT TinkerBell team and the ARL NS-CTA Knowledge Networks Construction task. She has served as a panelist for US Air Force 2030, and the Program Committee Co-Chair of several conferences including NAACL-HLT2018.