Lunch at 12:30pm, talk at 1pm, in 148 Fitzpatrick

Title: Lessons We Learned Towards Instruction Tuning: Data, Training and Evaluation

Abstract: Instruction-tuned language models have become popular recently due to their ability to solve any NLP task given a natural language instruction. Since the release of Llama and Alpaca models, a lot of research approaches have explored various key aspects of instruction tuning language models, including the data design, the training procedure and the evaluation protocol. In this presentation, I will briefly summarize some lessons we learned from instruction tuning papers in the past few months - what factors do researchers consider critical for tuning instruction-following models?

Bio: Zhihan Zhang is a third-year PhD student from the DM2 lab at Notre Dame, under the supervision of Dr. Meng Jiang. His recent research mainly focuses on large language models and instruction tuning, and he also has research experience in knowledge-augmented NLP and text retrieval. He has published multiple papers at top-tier NLP venues like ACL, EMNLP and TACL.