Lunch at 12:30pm, talk at 1pm, in 148 Fitzpatrick

Title: Boosting Language Models with External Knowledge

Abstract: Natural Language Processing benefits a lot from the recent trend of large-scale pretrained language models. However, the current language models lack access to external world knowledge about concepts, relations, and common sense. As a result, they often have underwhelming performance in NLP applications that requires world knowledge. In this talk, I will introduce our recent works on incorporating knowledge into language models. In particular, I will focus on how we achieved human parity on the popular CommonsenseQA benchmark using external knowledge attention to knowledge graphs, dictionaries and training data. We also demonstrate effectiveness of knowledge in multilingual commonsense, commonsense generation, open domain QA and dictionary-boosted pretraining.

Bio: Yichong Xu is a Senior Researcher in Knowledge and Language Team at Microsoft Cognitive Services Research Group. His main research interest covers a broad range of topics in Natural Language Processing and Machine Learning, including question answering, commonsense reasoning, knowledge graphs, multimodal learning and interactive learning. Prior to joining Microsoft, he obtained his Ph.D in Machine Learning from Carnegie Mellon University.