Decoding AI: Student Interactions and the Evolution of Qualitative Research

This presentation outlines a comprehensive, multi-institutional research study that investigates students' knowledge, affinity, use, and outcomes related to artificial intelligence (AI). As AI Technologies become increasingly integrated into study, work, and life, our research seeks to understand the extent and nature of this integration among students. 
The findings reveal that while AI is indeed permeating daily tasks, its application tends to be quite pedestrian in nature, contrary to the often sensationalized portrayals found in media and scholarly commentaries. This suggests a gap between public perception and the reality of AI’s current role in daily activities.
In addition to investigating student interactions with AI, this paper explores the evolving landscape of qualitative data analysis (QDA) in the context of recent advancements in large language models (LLMs). These models have demonstrated interesting capabilities in generating thematic analyses of data sets. Our study probes whether LLM analyses should be considered equivalent or complementary to traditional human-driven qualitative data analysis. To explore this question, we conducted an experiment involving two independent teams: one comprising human researchers and the other utilising an LLM-based text analysis tool. Both teams worked on a common set of interview transcripts.
Subsequent comparisons of their processes and outcomes provided valuable insights into the similarities and differences in how meaning is discerned from texts by humans and AI. Our findings highlight critical questions regarding the interpretative capabilities of LLMs compared to human researchers. 
This research highlights the potential for interactive AI tools to augment human qualitative data analysis by offering new perspectives and efficiencies. It is hoped that our study contributes to a deeper understanding of the practical applications of AI in both educational contexts and research methodologies, emphasising the nuanced role that AI currently plays in enhancing human capabilities rather than replacing them.