Final Thesis Information Science Undergradute Degree

Supervision of personal projects for the completion of the Bachelor studies in Information Science at Utrecht University. Examples of supervised projects:

• Fine-tuning a Language Model for Abstractive Dialogue Summarization

  • When chatting in real-time, online project groups often end up creating long-winded lists of sentences before heading toward a common goal or solution. Sometimes important information can get missed by group members. This creates a need for automatic chat summarization tools. In this thesis a natural language generation model (adopted from Z. Liu, Shi, and Chen (2021)) was fine-tuned, to generate a summary of the dialogue. The hyper-parameters used for fine-tuning the model were chosen by conducting small, exploratory experiments. Results show the final fine-tuned model is on par with current state-of-the-art dialogue summarization models. However, analysis of the outputs of the model shows it is not trustworthy enough for real-world application. We conclude that more research has to be done on incorporating dialogue-specific features to improve faithfulness of automatically generated summaries.

• Measuring Classroom Community During COVID-19 Higher Education via Microsoft Teams

  • In this project, we evaluated the effects of online education during the COVID-19 pandemic on the classroom community. We chose Microsoft Teams as the default channel since it is the official tool of Utrecht University. Looking at related literature, we set out a survey (N=28) amongst Utrecht University students where we were asked to compare the sense of classroom community between online and on-campus education. Our results showed that on-campus education significantly helped with improved classroom connectedness under the angle of) caring for each other, b) connectedness with others, c) isolation, and d) relying on others. Surprisingly, not all aspects of remote education negatively affected community connectedness in Dutch higher education since the sense of community was better online than on campus. We used the results as a starting point for a focus group (N=5) with a selection of the survey participants. From the results of the focus groups, we understood that physical proximity is key to increasing connectedness. Moreover, we noted that students are concerned with cameras online and that showing one’s face online is perceived as distracting and a violation of privacy. The focus group results confirm that students feel more a part of a community and less isolated online, which would be in line with the results of our survey. Other interactive tools that make the lecture engaging are a success, with online virtual school environments being the preferred online learning strategy.

• The effects of peer reactions within Microsoft Teams

  • Teamware, also known as collaboration platforms, and its use have grown significantly in the last few years. Microsoft Teams usage has surged as people have begun to work more from home. Microsoft Teams has several social media-like features, such as reactions to posts in the form of hearts, thumbs, smiles, and so forth, to convey a sentiment. People who use them, however, might be affected by their peers and their choice of reaction. This study investigates the impacts of peer reactions on Microsoft Teams. The results show that participants tend to change their choice of reaction only when another reaction receives the absolute majority of the votes. However, no change occurs when votes are distributed differently among reactions (e.g., when there is no clear majority or when two opposite sentiments receive similar vote counts). The study also shows that users from different generations react differently to posts and are affected differently by peer behavior.

• The Influence of Gender Representations on the Diversity of Self-Assembled Online Teams

  • Self-assembling team formation systems, where online users can select their teammates, are gaining research and industry interest. In today’s modern workplace, organizations are increasingly dealing with diversity’s many complex issues, especially gender identity. Research is beginning to investigate ways that technology can aid the formation of more diverse teams. Recent studies have shown that portraying the Diversity Info (DI) of online teammates improves diversity choices. However, more research is needed to evaluate the exact influence of different UI interventions on diversity. This thesis evaluates the effect of various task types and gender DI interventions on crowd users’ team members’ choices of User Interfaces (UI). We present two task types (creative and technical) on the team formation pages in combination with three conditions (control, gen-der icons, and gender labels). Our results indicate that displaying gender DI UI intervention can enhance diverse choices positively. Specifically, showing gender labels had a positive effect on the diversity of teammate choices, whereas showing icons did not show any significant results. Finally, task types also did not statistically influence the diversity of team member choices. Based on the findings, we suggest the use of gender labels on a self-assembling team formation system with the intent to make gender an explicit profiling attribute.

• The impact of priming and displaying diversity information on the formation of self-assembling teams

  • Online self-assembling team formation systems are becoming more widespread and hold the potential for connecting people to team members all over the world. The benefits of diverse teams, however, frequently remain unlocked as people tend to choose team members similar to them. To comprehend how these systems could increase the selection of more diverse team members, we experimented with an online self-assembling team system. We manipulated interfaces by displaying information about diversity and/or priming participants with counterstereotypes and all-inclusive multiculturalism. We expected the priming treatment to moderate the expected negative effect of displaying information on diversity. Neither treatments nor the combination of both treatments, however, affected the selection of team members in terms of diversity. Having a high need to belong, furthermore, did not significantly moderate or enhance the effects of the treatment conditions. These results contradict previous work. Factors that did predict selection behavior were level of education, order of appearance, and relevant functional background. Patterns of homophilic tendencies about ethnicity and region of origin confirmed previous work on people selecting similar others as team members. In light of these findings, we cautiously suggest that priming and displaying diversity information requires further research to yield more consistent results on diversity-enhancing mechanisms in practical implementations.

• Data Mining Twitter Feeds: Analysing thesentiment towards remote work during the COVID-19 Pandemic

  • This work explores the public sentiment of users from a collection of tweets regarding remote work and online collaborative tools. By gathering a variety of tweets using Tweepy, we analyzed them with Sentiment Analysis tools such as TextBlob and Vader NLTK. The results show that users shared a positive attitude toward working from home, especially when analyzing tweets containing words such as zoom, remote work, and webinar. In this exploratory study, we present the aggregate results of our analysis using sentiment analysis and word clouds. We further discuss the use and limitations of out-of-the-box tools for general sentiment analysis for fine-grained and topic-specific research.

• Towards prevention of unhealthy game usage for MMORPGs

  • Massive Multiplayer Online Role-playing Games (MMORPGs) tend to be frowned upon due to their addictive nature. Despite being only harmful to a small minority of players, researchers have indicated the need for preventive measures specific to this game genre. This study looks into addiction prevention within the game context through the player’s viewpoint. It considers three categories of preventive measures: educational, restrictive, and embedded. From an online study using a Discord Server, we collected accounts from 45 active MMORPG players who evaluated the proposed measures based on effectiveness and annoyance. The results showed that the participants perceived educational methods as effective instead of annoying. They also tolerated the inclusion of articles to educate the players on the risks of addiction and metrics such as the conceptualized addiction rating.

• The Effect of Moderation on Collaborative Problem-Solving Online

  • Moderation online, by a human or a chatbot, is a feature of text-based online chat that is becoming more common among online social community platforms. This study aims to determine whether the moderation is effective for collaborative problem-solving. Human and chatbot moderation is already effective in other areas, like education, entertainment, and virtual debates. In a series of experiments, participants solved the moon landing game together and collaborated using a chat application and drop-down menu. The study compared three conditions: Human moderation, chatbot moderation, and no moderation. The results indicate that human moderation is significantly better at improving team performance compared to the chatbot and the non-moderated conditions.