CSC News

Mining Data to Boost Collaborative Learning in Educational Games

NC State University researchers are applying data mining techniques to analyze the effectiveness of — and make real-time improvements to — educational games. They’re creating software tools that use real-time data to assess how well students develop collaborative problem-solving skills (CPS). These algorithm-based tools can enable educational games to be more responsive and effective for learning.

The software uses constraint-based pattern mining algorithms to assess and predict learning outcomes — techniques that have yet to be applied widely in an educational context.

“One of the attractive things about this framework is that it can evolve as more students play the game and the software has more data to draw on,” says Wookhee Min, co-author of a paper on the work and a senior research scientist at NC State. “In theory, that should allow us to fine-tune in-game interventions in order to improve learning outcomes even more.”

Currently, methods exist to help students develop collaborative problem-solving skills, but testing students to measure their performance can be disruptive.

The solution was to develop software that measured the cognitive aspects of CPS without intruding on game-playing. Based on a data-driven framework, the software can identify behavioral patterns.

In a study, 61 middle school students played EcoJourneys, a game-based learning environment designed to improve collaborative problem-solving emphasizing natural sciences. In teams, students try to figure out the cause of an illness spreading among the tilapia fish on a local farm. The game includes three quests, which are phases that support inquiry and collaboration. Students explore the game’s environment during the Talk & Investigate phase, interacting with non-player characters and viewing videos. In the Deduce and Explain phases, students apply their collaborative problem-solving skills.

In the study, researchers analyzed the raw data of student trace logs. Using the software and statistical modeling, they were able to find pedagogically relevant behavioral patterns and understand more about the effectiveness of how the game teaches CPS. 

The software can also modify the game in real time to adapt to student learning.

“For example, if students make a specific series of choices at one stage in the game, that may suggest that they are not grasping some key CPS concepts,” says Halim Acosta, first author of the paper and a Ph.D. student at NC State. “We could modify the game so that if students make that series of choices, the game changes in a way that emphasizes or reinforces those concepts.”

The research was presented at the Fifteenth International Learning Analytics & Knowledge Conference (LAK25), held March 3-7 in Dublin, Ireland. Additional co-authors from NC State include research scientist Seung Lee, senior research scientist Bradford Mott, and James Lester, the Goodnight Distinguished University Professor in Artificial Intelligence and Machine Learning at NC State and director of the university’s Center for Educational Informatics. The work was done with support from the National Science Foundation under grants 2112635, 1561486 and 1561655.

Adapted for the NC State Department of Computer Science; this article is based on a news release from NC State University

###

Note to Editors: The study abstract follows.

“Collaborative Game-Based Learning Analytics: Predicting Learning Outcomes from Collaborative Problem-Solving Behaviors”

Authors: Halim Acosta, Seung Lee, Wookhee Min, Bradford Mott and James Lester, North Carolina State University; Daeun Hong and Cindy Hmelo-Silver, Indiana University

Presented: March 3-7, 2025; 15th International Learning Analytics & Knowledge Conference (LAK25), in Dublin, Ireland

Abstract: Skills in collaborative problem solving (CPS) are essential for the 21st century, enabling students to solve complex problems effectively. As the demand for these skills rises, understanding their development and manifestation becomes increasingly important. To address this need, we present a data-driven framework that identifies behavioral patterns associated with CPS practices and can assess students’ learning outcomes. It provides explainable insights into the relationship between students’ behaviors and learning performance. We employ embedding and clustering techniques to categorize similar trace logs and apply Latent Dirichlet allocation to generate meaningful descriptors. To capture the temporal evolution of student behaviors, we introduce a graph-based representation of transitions between behavior patterns extracted using constraint-based pattern mining. We map behavioral patterns to a CPS ontology by analyzing how action sequences correspond to specific CPS practices. Analysis of semi-structured trace log data from 61 middle school students engaged in collaborative game-based learning reveals that the extracted behavioral patterns significantly predict student learning gains using generalized additive models. Our analysis identifies patterns that provide insights into the relationship between student use of CPS practices and learning outcomes.