Abstract: For medical students, virtual patient dialogue systems can provide useful training opportunities without the cost of employing actors to portray standardized patients. This work utilizes word- and character-based convolutional neural networks (CNNs) for question identification in a virtual patient dialogue system. While the CNNs perform well given sufficient training data, the best system performance is ultimately achieved by combining CNNs with a hand-crafted pattern matching system that is robust to label sparsity. In a retrospective study, we found that the combined system provided a 10% boost in system accuracy and an error reduction of nearly 50% as compared to the pattern-matching system alone. Subsequently, we replicated these results using live subjects, broadly confirming the findings from retrospective study but also highlighting important deficits. In particular, the hybrid approach continues to show substantial improvements over either rule-based or machine learning approaches individually, even handling unseen classes with some success; however, the system has unexpected difficulty handling out-of-domain questions. We attempt to mitigate this issue with moderate success, and provide analysis of the problem to suggest future improvements.

Short Bio: Dr. Michael White is a Professor in the Department of Linguistics at The Ohio State University. After obtaining his Ph.D. in Computer and Information Science from the University of Pennsylvania in 1994, Dr. White worked for eight years at CoGenTex, Inc., where he focused on developing practical applications of natural language generation (NLG) technologies during multiple SBIR and DARPA projects and industrial consulting engagements. In 2002, Dr. White crossed the pond to Scotland where he worked for three years as a Research Fellow at the University of Edinburgh, managing Edinburgh's effort on the COMIC dialogue system project. Since joining the faculty at OSU in 2005, where his work has been funded by multiple NSF and NIH grants, Dr. White has continued to work on NLG as well as applications of grammar-based paraphrasing, including for crowdsourcing training data for parsers and for data augmentation in the context of OSU's virtual patient dialogue system. Dr. White has also recently begun collaborating with Facebook AI, which is funding a new multi-year project on structure in neural NLG.