Lunch at 12:30pm, talk at 1pm, in 148 Fitzpatrick

Title: PaTAT: Human-AI collaborative qualitative coding with explainable interactive rule synthesis

Abstract: The use of AI assistance in data annotation has made significant progress. However, a specific type of annotation task, the qualitative coding performed during thematic analysis, has characteristics that make effective human-AI collaboration difficult. Informed by a formative study, we designed PaTAT, a new AI-enabled tool that uses an interactive program synthesis approach to learn flexible and expressive patterns over user-annotated codes in real-time as users annotate data. To accommodate the ambiguous, uncertain, and iterative nature of thematic analysis, the use of user-interpretable patterns allows users to understand and validate what the system has learned, make direct fixes, and easily revise, split, or merge previously annotated codes. This new approach also helps human users to learn data characteristics and form new theories in addition to facilitating the “learning” of the AI model. PaTAT’s usefulness and effectiveness were evaluated in a lab user study.

Bio: Simret Gebreegziabher is a second-year Ph.D. student at the University of Notre Dame, working with Dr. Toby Li at the SaNDwich (Science of AI at Notre Dame With Interaction between Computer and Human) lab. Her research focuses on the intersection of Human-Computer Interaction (HCI) and Artificial Intelligence (AI). Currently, she is investigating ways to design and comprehend tools that allow for effective data annotation through collaboration between humans and AI. Before starting her Ph.D., she received her Bachelor’s degree in Software Engineering from Addis Ababa University.