Abstract: Machine learning has become more and more prevalent in Human-Computer Interaction (HCI). However, its opaque and non-explainable decision-making process often makes it hard for people to trust the machine. In this work, we explored improving student experience on intelligent tutoring systems (ITSs) using hierarchical reinforcement learning (HRL) induced pedagogical policies and data-driven explanations of the HRL-induced policies. In three studies, we examined the impact of pedagogical policies and explanations on student learning independently and jointly. Overall our results showed that 1) the HRL induced policies could significantly improve students’ learning performance, and 2) explaining the tutor’s decisions to students through data-driven explanations could improve student-tutor interactions in terms of students’ engagement and autonomy. The results suggest that personalized RL decisions can be paired with human-authored explanations to achieve improved human-computer interaction outcomes, rather than using decisions or explanations alone.

Short Bio: Guojing is a graduating Ph.D. student in the department of computer science at North Carolina State University under Dr. Min Chi. He earned his M.S. in Computer Science from University of California, Riverside, and B.S. in Mathematics and Applied Mathematics from Beihang University (Beijing, China). His Ph.D. work focuses on improving the effectiveness of intelligent tutoring systems (ITSs). In this area, he mainly focuses on investigating the impact of decision granularity (problem level vs. step level) on student learning and applying hierarchical reinforcement learning (HRL) to induce pedagogical policies. His other work includes exploring student pedagogical decision-making and interpreting reinforcement learning induced pedagogical policies. His HRL work won the best paper award at AIED 2019.

Zhou, G., Azizsoltani, H., Ausin, M. S., Barnes, T., & Chi, M. (2019). Hierarchical Reinforcement Learning for Pedagogical Policy Induction. In S. Isotani, E. Millán, A. Ogan, P. Hastings, B. McLaren & R. Luckin (Eds.), Proceedings of Artificial Intelligence in Education (pp. 544-556): Springer International Publishing.