This post is part of our [work in] Progress series, an effort to share our preliminary findings on the impact of artificial intelligence in higher education to help the field move at the pace of technology.
Project Overview
Students’ sense of belonging is a critical component in their higher education success, yet little is understood about how to support belonging for online learners. The following research, conducted by WGU Labs, the research, development, and investment arm of Western Governors University (WGU), examines how a personalized belonging intervention — revised to address the experiences of online learners at WGU — potentially impacted students’ academic success. In collaboration with the WGU School of Technology, we found that students seemed to grasp the intervention’s message that the experience of struggle and frustration is normal and not an indication that the student does not belong in higher education. However, results show little impact of the intervention on learner outcomes, such as number of courses completed or retention into the second term. These limited impacts differ from prior research on similar interventions conducted in in-person university settings with predominately younger and first-time college students. This divergence from prior studies may reflect (1) a truly different dynamic related to belonging for online and/or older learners, (2) that WGU is a place that creates affordances for belonging, or (3), as we note below, could be due to a series of implementation challenges. We recommend exploring all possibilities further with additional students.
Key Takeaways
- The intervention was designed to communicate to students that not only were their feelings of worry or anxiety a normal part of the higher education experience, but that these experiences are shared by their peers. In analyzing students’ responses to the intervention’s quotes, we found that this intervention messaging was effectively communicated. When asked to reflect on what they learned from the quotes, intervention group students were more likely than control group students to remark on the fact that they are “not alone” and that they may have "anxieties or hesitations” about starting at WGU.
- Despite the intervention messaging seemingly being communicated well, analyses of student outcomes show there were no significant differences in student success and persistence between intervention and control group students.
- We discovered that the implementation of this intervention study was difficult. The intervention was distributed to all incoming students to our two programs of interest via their Orientation module. However, because it was not presented as a required activity, engagement was incredibly low. Future research will need to examine other implementation strategies for increasing uptake of the intervention.
Motivation and Goals
The rapid expansion of online learning and the rising popularity of artificial intelligence in higher education raises important questions about the significance of human connection in education and learning. Can online learning institutions, absent in-person interactions and opportunities, still create an environment where students feel like they belong? Without these face-to-face opportunities, the online learning environment may seem lonely or disengaging for students, especially those who are historically underserved or minoritized in higher education (for example, first-generation students or those from racial/ethnic minority groups). These online learners may be contending with societal biases or assumptions that they are not “cut out” for higher education. These disparities may be heightened in an online environment, where students have more asynchronous course engagement and fewer, if any, face-to-face interactions with their peers and instructors that could provide affirmation or support.
Previous research conducted by the WGU Labs team revealed that while maintaining a sense of belonging is just as important for online learners as it is for in-person learners, how belonging is created or communicated does differ based on learning modality. We found that belonging may derive more from recognition from the institution for online learners rather than from social or peer groups. Especially at WGU, which currently enrolls over 180,000 students, students report feeling seen amid all other WGU students when they are individually recognized or acknowledged for their achievements by a program mentor (PM) or course instructor (CI).
In the following research, we build off of our previous descriptive research on how WGU students define “belonging” to build out and test a personalized belonging intervention. The traditional belonging intervention involves the customization of student quotes that normalize the experience of frustration and struggle and highlight potential solutions for seeking help or support. Previous studies show that the intervention can support a sense of belonging in students, especially those from historically underserved groups, and can positively impact course and graduation outcomes. This research has historically been conducted at in-person institutions, and while the impact of the intervention has ranged, findings repeatedly show that there are positive effects of the intervention on buttressing a sense of belonging for students from groups that have historically received messages that they will struggle and underperform in higher education. One longitudinal follow-up of a belonging intervention study showed that students who completed the intervention were more likely than control group students to report that they had reached out and connected with a mentor during their time in college, which could be one reason for more improved outcomes. In our take on this research, we tested a WGU-specific belonging intervention, highlighting the common concerns or hesitations of incoming students and communicating to students that these are experienced by most WGU learners — and that there are ways of addressing these concerns.
Research Conducted
For this study, we tested the impact of a psychological intervention aimed at improving and/or deepening students’ sense of belonging. We distributed the intervention and complementary control condition to incoming students enrolled in the B.S. in Information Technology (BSIT) and B.S. in Cloud Computing (BSCC) programs at the WGU School of Technology. Students in the study were in the February or April cohorts for these programs. During their orientation module, we invited all students in the IT and Cloud Computing programs to complete the intervention or control surveys. One hundred total students consented to and participated in the study. We divided these 100 students evenly between the control and intervention groups.
Our sample, shown in Table 1, was well balanced by gender, race, education generation status, prior education, parental status, and program across the treatment and control groups (there were not statistical differences between intervention and control groups across the demographic variables).
In their Orientation module, students were presented with the following ‘Activity: Student Experience’ call-out box:
They were encouraged to learn more about the WGU student experience by clicking a link that randomly assigned them to view either the intervention or control
condition quotes (Table Y). We gathered these quotes from existing WGU students and attributed the quotes to existing WGU students and, depending on the condition, highlighted the following:
- In the intervention condition, students’ quotes encompassed the following themes:
- Worrying about the self-paced nature of WGU but then learning how to manage their time
- Not knowing how to connect with peers but then using Course Chatter (a course feature where students can discuss the course materials, ask questions, and share thoughts) to connect with classmates
- Nervousness at juggling school work with a full-time job but then connecting with their program mentor about strategies for scheduling their schoolwork
- Feeling lost within the large, online environment at WGU but then receiving personalized support/attention from their program mentor or course instructor
- Not knowing how to connect with peers but then participating in an online study group organized by their course instructor and finding like-minded peers who had the same struggles
- Feeling like they won’t be able to handle a tough IT certification but then reading tips on online community groups that made them feel more prepared and less alone
- In the control condition, students’ quotes shared themes similar to those above but did not address feelings of not belonging. Rather, the quotes focused primarily on the resources offered through WGU. This is similar to existing control conditions used in previous belonging intervention research. The main driver of the belonging intervention is that the quotes convey the sense that students are not alone in their worries, which are extremely normal and part of the transition into higher education, and the control condition allows the researcher to control for everything else the student is doing in the intervention (i.e., reading about student experiences, sharing resources, etc.).
{{table-of-quotes-1="/misc/rich-text-components"}}
More details about how the intervention quotes were constructed can be found in our Discovery Phase report.
After reading the quotes, students were presented with a series of survey questions that assessed such variables as belonging, self-efficacy, and intended engagement with institutional resources. Students then progressed with their first 6-month term. At the end of their first 6-month term (July 31 for the February cohort and September 31 for the April cohort), we assessed students’ academic success outcomes. These metrics serve as a proxy for determining the impact of the intervention. If we assume that the intervention increases students’ sense of belonging — which then increases students’ likelihood of reaching out for support and overcoming moments of failure — then it is possible that we would see this reflected in more improved academic outcomes.
- The percentage of students from each group who achieved On-Time Progress during their 6-month term
- Retention at 3 months
- Retention at 7 months
- The number of credit units students have remaining, controlling for number of transfer credits
- Percentage of assessments passed, given the number of assessments attempted
- Number of courses completed by the second month of their first term
Key Findings
Throughout the course of this project, we uncovered multiple insights to facilitate further research into belonging for online learners.
Finding 1: Intervention quotes normalize the experience of worry and anxiety when entering higher education
Before examining the potential impact of the intervention, we assessed whether or not students received the message conveyed in the intervention quotes about normalizing seeking help and reaching out for support and resources. In both conditions, after students read the student quotes, they were asked to reflect on the main takeaways from the quotes and what seemed helpful for a first-term WGU student to remember.
In line with the intervention goals, one noticeable difference across the groups was that students in the intervention group more frequently included emotion-related (e.g., nervous, anxiety, isolating) words in their reflections. This suggests that the intervention quotes effectively communicated the message that anxieties or worries about entering higher education were normal, and that respondents found this message particularly salient. For example, one student noted in their reflection, “It's OK to feel a little nervous and have some questions/doubts.” Intervention group responses were more likely to feature phrases where the students reminded themselves that they are “not alone” and that they may have "anxieties or hesitations” about starting at WGU, indicative of the messaging that was conveyed in their group’s quotes (that it is normal to feel nervous about starting but that they are supported by an incredible community of staff and faculty.)
Intervention group reflections:
- I think the main takeaways are that…[the students in the quotes] did not hesitate to contact their advisor for help, and that makes me feel better about what is to come. I know I am not alone and can do this!
- The main takeaway from reading about the stories is that feeling apprehensive about starting is OK. However, there are a myriad of resources and people to connect to that will help you on your journey.
- It's OK to feel a little nervous and have some questions/doubts. The keys are to remember that you are not alone and that there are resources to connect with for help, whatever your particular need is. You just have to communicate and seek answers to your questions.
- The main takeaway for me is that it is okay to make mistakes and/or fail an assessment. I think it is important to have a standard for yourself but not to be too hard on yourself.
Control group reflections:
- There are plenty of resources if you feel alone while taking courses.
- It doesn't matter where you are in life, WGU is flexible to fit into where you are.
- Be disciplined, organized, and use the WGU mentors, instructors, and community.
- Take your time and manage time wisely. Even with busy schedules.
In both control and intervention conditions, students noted that there is a wealth of resources at WGU that can support them throughout their learning journey, as expected given the content of the quotes. Students across conditions mentioned the presence of academic support staff, course instructors, and program mentors at WGU who can support them throughout their courses. Additionally, both groups’ students mention the flexible nature of the self-paced, competency-based education model of WGU. They remark on how the self-paced environment provides them the power to schedule their assignments and assessments at their own pace and more easily integrate completing their degree into their current life demands.
{{table-of-quotes-2="/misc/rich-text-components"}}
Finding 2: No differences in student pacing or retention outcomes between intervention and control group students
To assess the potential impact of the intervention on academic outcomes, we pulled the following outcomes from WGU’s institutional data:
- The percentage of students from each group who achieved On-Time Progress during their 6-month term (Fig. 1)
- Retention at 3 months and 7 months (Fig. 2)
- The number of credit units students have remaining, controlling for number of transfer credits (Fig. 3)
- Percentage of assessments passed given the number of assessments attempted (Fig. 4)
- Number of courses completed by the second month of their first term (Fig. 5)
The variety of metrics —courses attempted, assessments passed, credits earned, whether On-Time Progress was achieved— allows for us to investigate different proxies of student success at various points across their first term, providing a 6-month timetable for us to potentially observe the intervention’s effects.
Statistical comparisons between control and intervention groups do not show any significant differences between groups on any of the variables. This suggests that the impact of the intervention quotes was not strong enough to result in noticeable academic differences, though a more longitudinal investigation could examine whether the intervention may yield more noticeable outcomes over time.
Finding 3: Engaging students to complete the intervention and follow-up research was difficult
As stated in the “Research Conducted” section, students were invited to participate in this study through an optional activity within their Orientation module. Enrollment statistics show that over 400 students were newly enrolled in the B.S. in Information Technology and the B.S. in Cloud Computing for the February and April 2024 cohorts. However, only 100 students completed this optional activity. This sample size impacted our ability to conduct more rigorous statistical tests that would allow for a more granular understanding of the intervention’s potential impacts (e.g., was there a specific sub-group of students for whom this intervention was particularly impactful?) Historically, belonging interventions have seen more engagement and uptake when they are perceived to be a required part of the student onboarding experience — for example, when added to an Orientation checklist as an activity that students must complete.
Additionally, engaging students post-intervention to complete research surveys was difficult, despite leveraging communication from School of Business leadership and students’ own program mentors. We reached out to students’ program mentors to send out personalized communication to students to complete a short Qualtrics survey. Despite multiple rounds of communication, only a small percentage of students completed the survey. This survey would have allowed us to uncover whether or not the intervention quotes improved students’ perceived sense of belonging, despite this effect not showing up in the academic data. It is possible that while the effects of the intervention were not shown in the course outcomes, they could have impacted students more internally or psychologically, manifesting in higher levels of belonging and self-efficacy compared to the start of the term.
Next Steps
In this research study, we hypothesized that delivering a personalized belonging intervention to WGU School of Technology students might improve their academic outcomes and experiences at WGU. Though we did not uncover an impact on academic outcomes from the intervention, we did find promising differences between the control and intervention groups’ quote reflections. Specifically, we found that students interpreted the personalized intervention quotes as intended. Those individuals who read about the experiences of previous WGU students seemed to resonate with the messaging that it is normal to experience anxieties and frustrations, and that these moments of potential non-belonging are not indicative of them being unfit or incapable of succeeding in higher education. This serves as a promising foundation for continued research in this domain.
Though we did not find a measurable impact, we see the opportunity to build off this study in significant ways:
1. Delivering the intervention with a new population of students or at different pivotal moments
In future research, we could consider delivering the intervention to students outside of those who are entering their first term. While the belonging intervention research that has been conducted in higher education settings has typically been focused on first-term, first-time students, research in other settings has shown it to be generally effective in supporting individuals at pivotal moments that might challenge their sense of belonging. For example, some studies have shown these types of social-psychological interventions to benefit students when entering a particularly difficult course, especially one where their group is stereotypically perceived to underperform.
Another interesting student group to distribute the intervention to would be those who are weighing their decision to enroll at WGU. Perhaps students who are unsure whether or not they should enter WGU are plagued by feeling that they do not truly belong in a higher education setting. If this is the case, perhaps a belonging intervention delivered during the decision phase could empower them to proceed with their enrollment. Additionally, it is possible that delivering the intervention during a critical moment in a course (e.g., before a final assessment) could help reduce students’ fears or worries about their performance on the assessment, positively impacting their later performance in the course and persistence overall.
We understand that a limitation of this research was that it was only conducted with students in the School of Technology. It is possible that different results would have appeared if tested with students across the different WGU schools. Conducting this research more broadly across the various schools could yield more insights into the applicability of the current version of the belonging intervention.
2. Further personalization of the intervention
In our continuation of this work, we will plan to workshop the intervention and control quotes with students outside the School of Technology to better understand whether these experiences are generalizable across the schools and to identify what challenges or concerns might be specific to other schools at WGU. Research shows that the more the intervention can be contextualized to the specific institution and population, the more effective it may be. This highlights the importance of potentially engaging in more discovery work to unearth richer insights into the WGU student journey and their experiences of belonging.
3. Refining implementation and delivery strategy
In delivering the intervention, we encountered difficulties in engaging students to interact with the intervention materials. It is possible that embedding the intervention as a required element of the orientation process could yield higher engagement numbers. Whether it is requiring students to complete the intervention or giving students the option to opt out, rather than opt in, identifying a strategy for increasing engagement numbers will help generate the necessary sample for conducting rigorous analyses — especially analyses that will help determine, statistically, how the intervention impacted students in different subgroups.
4. Increasing understanding of how WGU creates opportunity
A pivotal finding that has come out of belonging research is that an essential precondition for the belonging intervention — and one that is more likely related to success — is that the learning environment creates the setting and affords the opportunities for students from historically underserved communities to feel like they belong. With this in mind, it would be a productive practice to assess how WGU communicates to students across various groups that they are welcome at and belong to the WGU community. How does this appear in their marketing materials? On the website? Is it reflected in the student groups or communities? By critically evaluating how opportunities for belonging are offered to students, we can better pinpoint spaces where belonging can be more actively communicated.
As more students enroll in asynchronous and online learning environments, it is imperative that we understand how to best support these students. Increasing and sustaining a high sense of belonging for students, especially those who might face more stigma or biases about their likelihood of success in higher education, has shown to be a critical indicator of future success — and a solution that is able to effectively support this effort could have a tremendous impact.
.png)