Forming Teams of Learners Online in a User as Wizard Study with Openness, Conscientiousness, and cognitive Ability
Published in Adjunct Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization, 2022
Recommended citation: Vinella, F. L., Koppelaar, S., & Masthoff, J. (2022, July). Forming Teams of Learners Online in a User as Wizard Study with Openness, Conscientiousness, and cognitive Ability. In Adjunct Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization (pp. 283-292). https://dl.acm.org/doi/abs/10.1145/3511047.3537660
Forming teams of learners is a task that presents numerous challenges for educators increasingly relying on automated tools to optimize the process. The problem increases in difficulty in online classroom settings, where educators have little familiarity with the students. In this work, we present a User as Wizard study where 108 online crowd participants formed four teams of three teammates each from a pool of twelve dummy learner profiles. The profiles contained information about the learners’ Conscientiousness, Openness, and cognitive ability levels. These attributes were derived from a pre-study with a smaller sample of crowd participants (N=52) rating the relevance of the Big Five personality traits and cognitive ability in team formation for educational purposes. The User as Wizard study shows that most people tend to form within (meaning most attributes of the teammates even out) and between (meaning the teams have similar attributes averages) balanced teams. It also shows that people perceive Conscientiousness and Openness as two of the most relevant personality traits when profiling learners for team formation. We compare these results to the probability of them being random and discuss the findings in the light of human-centered modeling of system designs and automation in education.