This research was produced using our Solutions Lab approach, which involves conducting small-scale, exploratory pilots in response to real learner challenges, allowing institutions to understand the efficacy of innovative solutions before scaling them. The following content is part of our [work in] Progress series, an effort to share our preliminary findings on the impact of artificial intelligence in higher education to help the field move at the pace of technology.
Introduction
The rapid advancements in and adoption of artificial intelligence (AI) in recent years have led to the exploration of AI technology within nearly every industry. Within education, AI technology has been investigated as a way to enhance learning through intelligent tutoring systems, chatbots, AI-assisted simulations, and support for students with disabilities. Previous research at Western Governors University, our parent organization, sought to understand how course instructors and students use AI tools to support their teaching and learning. This research revealed that students frequently self-serve for course support, often using resources outside of WGU (including AI tools) to provide the needed help. Furthermore, students and course instructors admitted to using these tools cautiously, engaging in these resources intentionally to support their teaching and learning.
This study builds on previous research by investigating the following research questions:
- How do students use AI tools to support their learning?
- How does students’ use (or lack of use) of AI tools impact their learning experience?
- How effective is AI in supporting the achievement of student learning outcomes?
Methodology
To answer these questions, we provided free access to ChatGPT Team to a select group of students enrolled in the Introduction to Programming in Python (C859/D335) course within WGU’s School of Technology. To recruit students for this pilot, all students assigned to a single course instructor for C859/D335 were emailed and invited to participate. From this initial invitation, 30 students expressed interest in the pilot and were asked to complete a consent form and accept a free ChatGPT Teams account. Among the 30 interested students, 18 completed these steps and were selected as pilot participants (see Appendix A). Over the two-month pilot, these students were invited to complete four bi-weekly surveys and participate in an interview and user testing session where they were asked to describe how they used ChatGPT to support learning and demonstrate their use of ChatGPT on an authentic Python problem. The findings described here come from 28 completed course surveys and interviews with three participants.
Executive Summary
Finding 1: Students' use of AI is goal-directed and oriented toward learning
Similar to previous research, students demonstrated intentionality in using AI tools to support their learning goals. As adult learners with competing work and family-related priorities, students sought to save time by turning to ChatGPT instead of the various course-provided materials. One participant observed, “If I'm just rushing it, like I had a long day at work, I'm just trying to get through this chapter in a certain course, and I just want to go to bed, but I want to finish this, then typically I’ll just type it in ChatGPT and move on.” More specifically, students saved time with ChatGPT by validating their understanding or helping them quickly overcome specific challenges. One student noted that he looked for “validation that I’m on the right track.” Another student used ChatGPT because he “had no idea where to begin… I just wanted it to start so I could understand what they were wanting out of it.”
With the goal of saving time, student use of AI was moderated by the potential impact these tools had on their learning and demonstrations of their knowledge and skill. Despite his use of ChatGPT, one student noted, “Knowing the material actually is more important to me than being able to get the answer faster.” He suggested that if he used ChatGPT more, “I’m not really learning.” Another student detailed conversations with friends in the field who described the importance of demonstrating skills without using AI technology. Speaking generally about AI in the workplace, he further described his desire to leave “a personal touch” on his work, an opportunity to “let your skill shine.”
Finding 2: Students use AI as a reference, teacher, and coach
Students’ descriptions of their use of ChatGPT within the course revealed distinct roles students placed on AI to support their learning. These roles highlight specific strategies students used with ChatGPT at different parts of the learning experience. For more specific student examples and use cases of ChatGPT, see Appendix B.
AI as a “Reference”
Using AI as a reference, participants sought quick, general information, examples, and answers to course-related questions or tasks. One participant mentioned using ChatGPT for a “quick download” of information in a similar way as they would when conducting research on Google. Other participants reported using ChatGPT to view examples of strategies for completing a task or receiving a solution or partial solution to a problem. Using given examples and whole or partial answers, students had access to a model of a potential solution or could identify the needed steps to arrive at a solution.
AI as an “Instructor”
Using AI as an instructor, students relied on ChatGPT to explain concepts, assess knowledge, troubleshoot errors, or optimize strategies and solutions, specifically in a more accessible way (e.g., using visuals, analogies) or using language that was more familiar. “Explain it to me like I’m five” was a strategy one student used with ChatGPT to explain complicated concepts.
Often, students would use ChatGPT to assess their performance and check for understanding. Within a programming course like Python, this often meant checking the student’s code for possible errors or troubleshooting errors they had received. Some students also prompted ChatGPT to explain the reasons for the error and how to fix it, enabling ChatGPT to provide instructional guidance for future application.
Students were aware of the many ways to code solutions to problems, and some students used ChatGPT to identify and understand more efficient or effective ways of programming solutions. After inputting a working solution to a problem, one student prompted ChatGPT, “I got this solution. Is there a quicker or better way to do this?”
AI as a “Coach”
Using AI as a coach, students sought to improve their experience and learning process in the course and move beyond moments of mental frustration. For some students, especially those new to programming, these frustrations began almost immediately as they tried to understand this new language (i.e., Python). ChatGPT was a resource for alleviating the overwhelming feeling of burden for students by helping them know how to start, what the desired outcome might look like, or how the complex task may be broken down into smaller components that were easier to understand. In other situations, moments of being stuck occurred while learners were immersed in a problem but could not see a way through the challenge. In each of these cases, ChatGPT was a resource for hints, nudging students beyond the moment of difficulty to improve their overall experience in the course.
Students used these distinct roles repeatedly and interchangeably throughout the learning experience. No student relied exclusively on one role; each role of AI seemed to provide value and help at different parts of students’ learning journey.
Finding 3: Anecdotally, students reported improved learning and experience with AI
Student reports suggest that having access to ChatGPT generally improved their learning and experience in the course. In a survey asking participants to rate the level of helpfulness of AI on their learning, the average response was 3.39 (out of 4), suggesting that students evaluated AI as “moderately helpful” to “very helpful.” However, how much these tools contributed to learning metrics (e.g., assessment score, pass rate) is unclear.
Similarly, students noted a positive impact of access to ChatGPT on their course experience, but it is unclear how much AI impacted course experience metrics (e.g., course satisfaction, on-time progress). One participant suggested that having access to ChatGPT helped him to pass the course faster and with less overall difficulty. Interestingly, another participant indicated that having access to ChatGPT promoted more “light bulb moments,” a new opportunity to think differently about the content and the different tools that are available.
Next Steps
While these insights emerge from a small group of students, this study illuminates opportunities for further research and student support. These findings show that students are intentional in their use of AI for learning, assigning distinct roles to AI through targeted and specific interactions with AI. However, more research is needed to understand how these findings may generalize across various courses, programs, and colleges at WGU — and beyond. For example, do the roles of AI stay consistent across courses and programs? Does non-technical subject matter (e.g., arts, humanities, social sciences) encourage different techniques for using AI? Do students in different courses and programs manifest similar or different help-seeking behaviors from AI technology?
Additionally, students in this study were interested in continuing their use of AI in future learning, suggesting opportunities for more consistent integration of AI within the learning experience. When and where might these AI tools be most effectively integrated? What resources are needed (e.g., job aid, video) to support students’ effective use of AI technology? How might these resources be provided in a way that supports the development of AI literacy expected by employers in the student’s selected industry?
Through ongoing exploration of the use and application of AI technology within the learning experience, WGU Labs aims to continue creating pathways to opportunity by supporting student attainment of industry-recognized skills and competencies.
Appendix A: Participant Descriptives
Appendix B: ChatGPT Use Cases
