Lecturer's blog 18: Insights into Thesis Students' Use of AI Tools by Martijn Keizer

Over the past few weeks, I have collected data on the use of AI tools by FEB students who are currently writing their thesis. This lecturer’s blog seemed an excellent podium to share the main results and reflect upon what they might mean.

Collecting information on AI use among thesis students is potentially interesting and important as it begins to fill a gap in our understanding of how widespread the use of AI tools is at FEB. While individual thesis supervisors may have anecdotal evidence of AI tool usage among their students, there's a lack of a clear picture of the scale of this phenomenon. This survey, which combines descriptive information with questions about the motivations behind AI tool usage, offers a first step toward mapping AI tool adoption among FEB students.

Survey Overview

A total of 240 students, split between Bachelor (88 participants) and Master (152 participants) levels, completed a survey on the prevalence and nature of AI tool usage in thesis writing. The survey was distributed by thesis coordinators of all BSc and MSc programs at FEB via Brightspace. It contained questions about students' likelihood of employing AI tools, the specific tools in question, motivations behind their selection, and the anticipated impact of the use of AI tools on the quality and efficiency of thesis work. Data was collected in February 2024; all respondents had just started work on their thesis

Detailed Results

Likelihood of AI Tool Usage

A striking 90.6% of respondents indicated that they considered or were likely to use AI tools when working on their thesis. Bachelor and Master students scored almost identically, with 89.5% of Bachelor students indicating a willingness to use AI tools and 91.6% of Master students doing so. These results demonstrate a clear openness to utilizing AI tools in thesis work.

Purposes for which students expect to use AI Tools

All Students who indicated a willingness to use AI tools were asked to tick boxes in a list of potential purposes for which they might use such tools. The main purposes for which students expect to use AI tools are text beautification (defined as ‘the use of a (Gen)AI tool to improve the quality of written text) and brainstorming (defined as ‘using a (gen)AI tool to come up with ideas’); the vast majority of students indicated they intended to use AI tools for both of these two purposes. Somewhat less widespread, but still often selected, were the use of AI tools for finding (scientific) references and summarizing scientific articles.

Reasons for using AI tools

The desire to improve thesis quality was a significant motivator, with many students indicating that they want to use AI tools for text beautification and for brainstorming to improve its quality. Saving time on thesis work was another key reason that was often mentioned, particularly in association with summarizing scientific articles and finding scientific references. Finally, students also cited overcoming personal skill gaps and the ability to verify the quality of their own work as reasons for potential AI tool usage.

Reasons Against Using AI Tools

The small group of students who indicated they would not consider the use of AI tools when working on their thesis essentially gave two main reasons for refraining from doing so. One group indicated concerns about their ability to use AI tools well. The other students listed a desire for the thesis to be all their own work as the main reason for not considering the use of AI tools. Interestingly, concerns related to privacy or data safety seemed to play no role in students’ considerations regarding the use of AI tools.

Anticipated effects of AI tool usage on thesis quality and time investment

Students were asked to indicate their expectations regarding the quality of their thesis on a five-point scale. The images below reveal that at both Bachelor and Master level, the majority of students anticipate an improvement in the quality of their thesis as a result of the use of AI tools. Master students are slightly more optimistic than Bachelor students, with 85.6 percent of Master students anticipating an improvement in the quality of their thesis as a result of the use of AI tools.

Many Bachelor and Master students expect that the use of AI tools will reduce the amount of time they will spend on their thesis. Bachelor students expect a bigger impact of AI tools on their time investment; this may be indicative of more optimism among this group about the possible impact of AI tools on their work, or of a stronger emphasis on saving time (vs improving quality) among Bachelor students compared to Master students.

AI Tools Mentioned

ChatGPT emerged as the most mentioned AI tool, with 160 references, indicating its perceived utility in aiding the thesis writing process, followed by Grammarly, which garnered 10 mentions for its role in improving written content. A host of other (often free) AI tools was mentioned once or twice.

Paid Subscriptions

A final set of questions focused on paid subscriptions for AI tools. Students displayed a general hesitancy towards paid subscriptions, as only 30 students reported having invested in such services. ChatGPT 4 was by far the most popular paid-for service. This suggests that for now, most students rely on free services and are not using the full capabilities of the paid-for versions of these tools.

Conclusions

Students are quickly adopting free AI tools to assist them in writing their theses. They expect that these tools will improve the quality of their work while also saving them time and effort. These results differ quite significantly from the results of similar surveys at other Universities that have been published recently. The discrepancy is likely at least partly the result of the increasing familiarity of many students (and others) with GenAI models over time. As more people become aware of the possibilities of these tools and as the capabilities of these tools continue to grow, the number of users is likely to continue to grow as well.

The increasingly broad adoption of AI tools by students in thesis writing raises serious questions about the skills and knowledge students need in order to do well and the assessment of their work. I will not attempt to answer these questions in this blog, but I am hoping to contribute to finding at least some of the answers in my STQ project on the future of the master thesis. This project will formally start in September; anyone who would like to know more or is interested in sharing his/her ideas about this topic is more than welcome to contact me at m.keizer rug.nl.

This concludes this blog. I would like to pass the torch to Tri Tran from the Department of Operations, who will focus on the potential impact of GenAI on course design.

For the sake of readability, I did not insert specific survey questions or references to scientific literature. However, both the full survey and more information on recent scientific papers on students’ use of AI tools are available upon request, so feel free to contact me if you want to know more.