A Grounded Theory Investigation of Basic Psychological Needs : A Grounded Theory Investigation of Basic Psychological Needs
Theory as a F y as a Framework for E amework for Effective Mental Health Ref e Mental Health Referrals in
Collegiate Sport
Luna Ugrenović
West Virginia University
Follow this and additional works at: https://researchrepository.wvu.edu/etd
Part of the Other Psychology Commons
Recommended Citation
Ugrenović, Luna, "A Grounded Theory Investigation of Basic Psychological Needs Theory as a Framework
for Effective Mental Health Referrals in Collegiate Sport" (2024). Graduate Theses, Dissertations, and
Problem Reports. 12694.
https://researchrepository.wvu.edu/etd/12694
This Dissertation is protected by copyright and/or related rights. It has been brought to you by the The Research
Repository @ WVU with permission from the rights-holder(s). You are free to use this Dissertation in any way that is
permitted by the copyright and related rights legislation that applies to your use. For other uses you must obtain
permission from the rights-holder(s) directly, unless additional rights are indicated by a Creative Commons license
in the record and/ or on the work itself. This Dissertation has been accepted for inclusion in WVU Graduate Theses,
Dissertations, and Problem Reports collection by an authorized administrator of The Research Repository @ WVU.
For more information, please contact researchrepository@mail.wvu.edu.
A Grounded Theory Investigation of Basic Psychological Needs Theory as a Framework for
Effective Mental Health Referrals in Collegiate Sport
Luna Ugrenović, M.S., M.A.
Dissertation submitted
to the College of Applied Human Sciences
at West Virginia University
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy in
Sport, Exercise, and Performance Psychology
Dana K. Voelker, Ph.D., Chair
Ashley Coker-Cranney, Ph.D.
Johannes Raabe, Ph.D.
Monica Leppma, Ph.D.
School of Sport Sciences
Morgantown, West Virginia
2024
Keywords: interpersonal behaviors, need supporting, need thwarting, sport support personnel,
teammates, student-athletes
Copyright 2024 Luna Ugrenović
ABSTRACT
A Grounded Theory Investigation of Basic Psychological Needs Theory as a Framework for
Effective Mental Health Referrals in Collegiate Sport
Luna Ugrenović
Using a grounded theory approach, this study explored collegiate athletes’ mental health referral
experiences and identified mechanisms explaining those experiences based upon perceptions of
psychological need fulfillment and thwarting according to Basic Psychological Needs Theory
(BPNT). Interviews were conducted with a purposive, maximum variation sample of 14 current
and former collegiate athletes who were referred to mental health services by support personnel
or other athletes. Analyses yielded three overarching theoretical categories/properties: prereferral factors, proactive and reactive referral conversations, and referral outcomes. Referral is
often understood as an isolated conversation in which athletes are provided mental health
resources in response to a crisis. Challenging this notion, the present study showed that proactive
referral as a preventative approach, as well as intentional understanding athletes’ pre-referral
experiences, were central to supporting athletes’ relationship with their mental health. Generally,
referral experiences that resulted in helping athletes disconnect their mental health concerns from
understandings of their self-worth, and ultimately pursue mental health services with hope for
recovery, were linked to psychological need satisfaction throughout the referral process (e.g.,
autonomously choosing mental health services, relating to the referrer through a similar
experience, mastering skills for pursuing therapy). Yet, in cases of positive referral experiences,
some athletes re-internalized mental health stigma upon re-engaging with unsupportive sport and
familial environments. Need thwarting amid the referral conversation was generally associated
with poorer outcomes, in addition to cultural environments that deterred help-seeking. Together,
the data suggest BPNT is a promising framework for effective referral guidance, though
psychological needs are met uniquely for each athlete based upon their characteristics and
context.
MENTAL HEALTH REFERRALS IN SPORT iii
Acknowledgments
I cannot thank enough everyone from both sides of the ocean who have supported me on
this journey. I would like to express my deepest gratitude to Dr. Dana K. Voelker, my advisor
and committee chair, for her invaluable patience, guidance, and support. Her passion for the field
and dedication to the highest excellence has pushed me to reach my potential. This endeavor also
would not have been possible without my committee, who generously provided their knowledge
and expertise. I thank Dr. Ashley Coker-Cranney for her methodological expertise and guidance
that made this ambitious study possible. I thank Dr. Johannes Raabe for challenging me to reach
the highest research standards and offering his theoretical expertise. I thank Dr. Monica Leppma
for reminding me to trust my intuition and to practice self-compassion. I would also like to
deeply thank my research assistant, Verena Zaisberger, for her dedication and perspective during
each phase of the study – her curiosity and hard work were pivotal in creating the final product
of this project.
I am also so grateful for my colleagues and friends who read drafts, offered feedback, and
were my moral support. Without them, my journey would not have been nearly as meaningful
and fulfilling. I will be forever grateful to Dr. Tommy Minkler, Dr. Blake Costalupes, Dr. Sofía
España Pérez, Dr. Zenzi Huysmans, Dr. Karly Casanave Phillips, Dr. Andrew Chip Augustus,
Kim Tolentino, Jaxson Judkins, Kayla Hussey, Hannah Miller, Sarah Sadler, Hannah SilvaBreen, and Lindsey Leatherman.
I thank my family, especially my parents, Maja and Vladan, and my sister, Mia. Their
belief in me has kept my spirits and motivation high, while their unconditional love has offered
me a sanctuary when I needed it. Lastly, I want to express special thanks to my husband, Salman,
whose tireless support, love, and selflessness helped me overcome each and every obstacle along
the way. Our shared passion for the topic and many long late-night discussions about the
grounded theory have inspired me and reminded me of the opportunity to impact others with my
research. I thank him for sharing this journey with me every step of the way.
MENTAL HEALTH REFERRALS IN SPORT iv
Table of Contents
Introduction and Literature Review…………..…………………………………………….……..1
Basic Psychological Needs Theory………………………………………………………..4
Methods and Materials…………………………………………………………………………….7
Research Approach and Philosophical Assumptions..…...…………………...…….……..8
Methodological Approach………………………………..….………….………...………9
Participants………………………………………………………………………..……...10
Materials…………………………………………………………………………………11
Demographic Survey…………………………………………………………….11
Interviews………………………………………………………………………...11
Procedures………………………………………………………………………………..12
Data Analysis…………………………………………………………………………….14
Methodological Rigor……………………………………………………………………18
Results……………………………………………………………………………………………19
Pre-Referral Factors….…………………………………...……………………………...20
Participants’ Identities and Culture……………………………………………....20
Participants’ Family Environments………………………………………………22
Participants’ Sport Environments……………………...………………………...23
Reactive and Proactive Referral Conversations………………...…………………....…..25
Psychological Need Satisfaction and Thwarting………...………………………26
Autonomy Satisfaction and Thwarting…………...……………………...26
Relatedness Satisfaction and Thwarting………..….…………………….28
Competence Satisfaction and Thwarting…………….…..………………30
Referral Outcomes……………………………………………………………………….30
Perceptions of Self as Context……………………………………...….……...…31
Perceptions of Self as Content…………………………...……………………....32
Discussion………………………………………………………………………………………..36
Practical Implications…………………………………………………………………….42
Limitations and Future Directions………………………………………...……………..44
Conclusion……………………………………………………………………………….46
References………………………………………………………………………………………..48
MENTAL HEALTH REFERRALS IN SPORT v
Appendix A: Extended Literature Review…………………………………………….……...….59
Collegiate Athlete Mental Health……………………………………………...……...…60
Mental Health Referrals………………………………………………………………….64
Mental Health Referrals in Sport………………………….……………...……...70
Rates of Mental Health Referrals in Collegiate Sport…..…………...…..72
Possible Barriers to Successful Mental Health Referrals in Sport……….74
Mental Health Referrals in Sport – Best Practices……………………….78
Basic Psychological Needs Theory………………………………………………………84
Basic Psychological Needs Theory and Collegiate Athletes…………………….87
Basic Psychological Needs Theory, Well-Being, and Mental Health….....……..90
References………………………………………………………………………………..92
Appendix B: Extended Methods and Materials………………………………………….…......106
Researcher’s Positionality and Background……………………………………………106
Appendix C: Recruitment Flyer………………………………………………………………...109
Appendix D: Cover Letter………………….………………………...………………....……...110
Appendix E: Demographic Survey……………………………………………..………………112
Appendix F: The Final Version of The Interview Guide…………………………….…………115
MENTAL HEALTH REFERRALS IN SPORT 1
Introduction
Mental health concerns are rising among collegiate athletes (Brown et al., 2021). Data
suggest the prevalence of stress, anxiety, sadness, mental exhaustion, anger, and sleep difficulties
are 1.5 to 2 times higher than pre-2021 reports (National Collegiate Athletic Association
[NCAA], 2022). Collegiate athletes experience signs and symptoms of various mental health
concerns (e.g., Brown et al., 2021; Gavrilova & Donohue, 2018) at greater frequencies than their
same-age, non-athlete peers (e.g., Donohue et al., 2018; Li et al., 2017). Despite collegiate
athletes’ reported frequency of mental health concerns, research suggests most do not pursue
mental health services. For example, the NCAA Well-Being Study (2022) found that less than
half of the collegiate athletes surveyed reported feeling comfortable seeking help from an oncampus mental health professional. There are various barriers to collegiate athletes’ willingness
and ability to pursue mental health services, such as stigma, limited knowledge of the supports
and resources available, negative help-seeking beliefs, concerns about confidentiality, and
perceived lack of time (e.g., Cox et al., 2017; Hilliard et al., 2020; Moore, 2016). To address this
problem, understanding how best to connect collegiate athletes with mental health services, in
ways they feel comfortable and motivated to pursue those services, is gravely needed.
An important step in the mental health help-seeking process, and that can be investigated
to address collegiate athletes’ growing mental health concerns, is referral. Defined as the process
of directing others to appropriate supports and resources to address their mental health needs
(Substance Abuse and Mental Health Services Administration [SAMHSA], 2015), referral is
often made by peers or support personnel in collegiate athletes’ immediate performance
environments, such as academic advisors, strength and conditioning coaches, sport coaches,
sport dieticians, mental performance consultants, physiotherapists, physiologists, and athletic
MENTAL HEALTH REFERRALS IN SPORT 2
trainers (Cormier, 2014; Daltry et al., 2021; Neal et al., 2013; 2015; Sebbens et al., 2016; Van
Raalte et al., 2015). Unfortunately, support personnel report they are neither comfortable nor
confident in their ability to communicate with, refer, and support athletes experiencing mental
health concerns (Gayman & Crossman, 2006; McArdle et al., 2018; Moreland et al., 2018;
Sandgren et al., 2022). In a qualitative study, support personnel reported feeling anxious,
frustrated, underprepared, and unskilled to refer athletes, citing a critical need for guidance on
what and how to communicate with athletes experiencing mental health difficulties (McArdle et
al., 2016). Gayman and Crossman’s (2006) qualitative study found mental performance
consultants rarely discussed the possibility of referrals to mental health services during intakes,
did not maintain an updated referral list of available providers, and did not follow-up with the
athlete clients if a referral was made. Unsurprisingly, this discomfort with mental health
conversations is compounded by collegiate athletes also feeling the effects of this uncertainty
(Arthur-Cameselle et al., 2012). For example, Cutler and Dwyer’s (2020) study of 158 collegiate
athletes found that only half believed their coaching staff could offer support in an emotional
crisis. Clearly, a lack of preparedness and comfort with mental health has led to the avoidance of
referral-related behaviors among support personnel and is also impacting athletes.
Collectively, these data suggest that although support personnel can play a key role in
referrals given their proximity to and rapport with collegiate athletes (e.g., Daltry et al., 2021;
Neal et al., 2015; Sebbens et al., 2016), barriers to effective communication with athletes around
mental health concerns may impede the referral process, perpetuate negative help-seeking
attitudes and stigma, and in turn make it exceedingly difficult for collegiate athletes to seek
appropriate and timely help when needed. These communication barriers are not just limited to
the exchange between support personnel and athletes, as many athletic departments also do not
MENTAL HEALTH REFERRALS IN SPORT 3
sufficiently discuss the referral process, suggesting structural difficulties that some coaches have
described as “broken” and “dismantled” (Eckenrod et al., 2023, p. 7). Accordingly, support
personnel must be better equipped to support collegiate athletes through the development and
implementation of referral guidelines that ease and facilitate communication while empowering
athletes to pursue mental health supports and resources. This need extends to athlete peers, as
some literature suggests that, with adequate training, college student peers generally, and athlete
peers specifically, can act as helpful sources of mental health referrals (e.g., Kalkbrenner et al.,
2019; Van Raalte et al., 2015).
Yet, extant referral guidelines designed for athletes are limited. Vella at al.’s (2021)
systematic review and meta-synthesis of mental health position statements in sport found the
overall quality of the guidelines to be low. Specifically, although many policy statements discuss
the importance of referrals, they lack clear referral guidelines, such as when, how, and to whom
to refer. Guidelines solely focused on policy and procedure (i.e., what to do) but not process (i.e.,
how to do it) are limited in their effectiveness (Tod & Anderson, 2014) as they fail to support the
application and implementation of their referral policies and procedures to specific contexts. This
not only refers to the sport environment context but also to collegiate athletes’ identities and
cultural backgrounds, as collegiate athletes working with support personnel reportedly value
when their identities beyond sport are acknowledged and honored (Bejar et al., 2019). Further, as
culture influences the likelihood of engaging with mental health care in unique ways (Abdullah
& Brown, 2011), athletes’ identities and cultural contexts may also impact referral outcomes.
More recently, the NCAA (2024) released a mental health best practices consensus document,
yet their recommendations are also on referral pathways and not processes. Although such efforts
MENTAL HEALTH REFERRALS IN SPORT 4
are timely and important, there remains a need to delineate specific steps and actions that support
personnel and other athletes can utilize when engaging in the mental health referral conversation.
Recommendations that do explicate process are largely based on accounts of experienced
practitioners and include: (a) building rapport with athletes and communicating care for their
well-being; (b) facilitating a team approach to athlete care; (c) preparing athletes for referrals; (d)
showing support and normalizing mental health concerns; (e) referring in (i.e., inviting therapists
to meetings with athletes and support personnel); (f) explaining reasons for referrals; (g)
describing what working with therapists entails; (h) following up with athletes; and (i) being
prepared to offer alternative coping options and to revisiting referral after giving athletes time to
consider the recommendations (Andersen et al., 1994; Neal et al., 2013; 2015; Tod & Andersen,
2014; Van Raalte & Andersen, 2014). Such practitioner-informed guidelines can be further
strengthened by centering athletes’ voices on effective and ineffective referral experiences,
considering athletes’ identities and unique needs, and studying the interpersonal components
with referrers that may affect athletes’ willingness and ability to pursue mental health
recommendations. Further, using well-established conceptual frameworks to explore and explain
the referral process would provide additional structure to future research in this area, particularly
with respect to predicting athletes’ willingness to engage in the recommendations they are
provided, while providing clarity and focus to key targets of intervention that would improve the
referral experience.
Basic Psychological Needs Theory
A promising framework for understanding how to facilitate effective referrals for
collegiate athletes, is basic psychological needs theory (BPNT; Ryan & Deci, 2017), a wellresearched theoretical guide for successful interpersonal interactions toward future behavior.
MENTAL HEALTH REFERRALS IN SPORT 5
BPNT may be especially helpful in this context as it has been widely used in sport research to
explore the impact of interpersonal behaviors on motivation (e.g., Raabe & Zakrajsek, 2017).
Specifically, as one of the mini theories within the larger framework of self-determination theory
(SDT; Ryan & Deci, 2017), BPNT has helped explain antecedents to initiating and maintaining
behaviors in sport (e.g., Raabe et al., 2016), work (e.g., Parfyonova et al., 2019), health care
(e.g., Ryan at al., 2008), and other settings.
According to BPNT, an individual’s innate and basic psychological needs are autonomy,
competence, and relatedness, all three of which are influenced through social interactions and are
considered essential for psychological growth (Ryan & Deci, 2017). Autonomy refers to
perceptions of self-governance and volition and the degree to which one values these behaviors.
While autonomy support involves conveying value of one’s perspective and encouraging selfdirectedness over one’s decisions, autonomy controlling behaviors involve the use of external
controls, like rewards and punishments, that promote need thwarting and undermine motivation.
Competence refers to perceptions of self-mastery in one’s interactions with their environment.
Supporting another’s need to feel competent involves providing education, guidance, feedback,
and structure to assist in the development of mastery and efficacy. Conversely, thwarting
another’s need to feel competent involves highlighting faults and lack of abilities, demonstrating
doubt in the ability to improve, and discouraging engagement in difficult tasks. Finally,
relatedness pertains to a sense of belonging in which one feels connected to, and valued by,
others. With respect to relatedness, need support constitutes genuine interest, care, acceptance,
and companionship from others, whereas need thwarting constitutes treatment from others that is
distant, emotionally unavailable, exclusionary, and characterized by a failure to listen. As others’
behaviors influence the extent to which one perceives these needs to be satisfied, or thwarted,
MENTAL HEALTH REFERRALS IN SPORT 6
this in turn determines the likelihood of initiating and maintaining a behavior and informs
wellbeing. Although specific ways basic needs are satisfied or frustrated differ based on context,
culture, and social setting, they are considered to universally apply to all humans (Ryan & Deci,
2017). In a recent meta-analysis (Slemp et al., 2024), perceived need satisfaction, coupled with
interpersonal support, was associated with optimal motivation, wellbeing, and performance.
Conversely, negative interpersonal behaviors informed perceptions of need thwarting that, in
turn, were associated with ill-being, poorer performance, and diminished motivation.
Researchers have used BPNT to understand the wellbeing and mental health of diverse
groups of young adults (e.g., Akbag et al., 2017; Campbell et al., 2018; Cantarero et al., 2021),
including athletes. For instance, in a sample of over a thousand collegiate students from four
countries (i.e., Belgium, China, USA, and Peru), Chen at al. (2015) found need-satisfaction
predicted well-being, and need-thwarting predicted ill-being, irrespective of students’ nationality,
their reported importance of each need, and their willingness to satisfy them. This evidence
highlights the utility of a BPNT framework in understanding mechanisms for supporting mental
health among college students. BPNT has been also used in collegiate sport to explore athletes’
fulfilment and frustration of the three basic psychological needs in both academic and athletic
settings (e.g., Raabe at al., 2022; Raabe & Readdy, 2016), and coaches’ and teammates’ impact
on collegiate athletes’ need-satisfaction (e.g., Raabe & Zakrajsek, 2017).
Some studies have specifically examined the role of support personnel (e.g., coaches and
athletic trainers) in injured athletes’ need satisfaction and thwarting during their recovery and
return to sport. Bejar et al.’s (2019) study of NCAA Division I collegiate athletes with an injury
found greater motivation during injury rehabilitation when they perceived their athletic trainers
supported their needs for autonomy, competence, and relatedness (Bejar et al., 2019).
MENTAL HEALTH REFERRALS IN SPORT 7
Specifically, participants reported that athletic trainers facilitated their motivation to engage in
rehabilitation recommendations when they provided (a) information, clear expectations and
goals, encouragement, and reassurance (facilitating their competence); (b) scheduling availability
and flexibility, focus on both health and performance goals, receptivity to feedback, and choice
in treatment (facilitating their autonomy); and (c) approachability, rapport-building, and curiosity
about athletes’ lives beyond sport (facilitating relatedness; Bejar et al., 2019). The findings of
this study suggest BPNT may be a useful framework for investigating the role of support
personnel in collegiate athletes’ motivations to pursue recommendations around their care.
Specifically, behaviors of support personnel and athlete peers may similarly influence collegiate
athletes’ motivation to follow-through on referral recommendations by interacting with their
psychological needs and, in turn, impacting their wellbeing.
Situated within their identities and cultural contexts, this study explored (a) collegiate
athletes’ experiences receiving mental health referrals from support personnel and their athlete
peers, and (b) the mechanisms, informed by perceptions of psychological need supporting and/or
thwarting, that explained the experiences and their associated outcomes. The findings are used to
inform theoretically-grounded, data-driven, and athlete-centered recommendations for referral in
collegiate sport environments.
Methods and Materials
Research Approach and Philosophical Assumptions
The researchers adopted a multiparadigm approach integrating constructivist and critical
realist research paradigms (Bogna et al., 2020; Voelker et al., 2024) to fully achieve the research
aims. As proposed by Bogna et al. (2020), a constructivist paradigm allowed the researchers to
explore the domain of the empirical, as guided by (a) relativist ontology whereby researchers
MENTAL HEALTH REFERRALS IN SPORT 8
assumed there were multiple and socially constructed realities (Sale et al., 2002); (b) subjective
epistemology in that meaning was believed to be socially- and culturally-informed through
interactions with one’s environment (Moon & Blackman 2014), and (c) the axiological aim was
to understand and communicate co-constructed realities between the researchers and the
participants (Biedenbach & Jacobsson, 2016). Toward the first study aim, a constructivist
paradigm was used to first give voice to participants themselves, and in that way, understand
their mental health referral experiences and meaning. Specifically, participants’ referral beliefs
and experiences were perceived as products shaped by social and cultural contexts associated
with their unique identities and collegiate sport environments and were considered as multiple
and individual truths.
A critical realist paradigm allowed the researchers to explore the domain of the real
(Bogna et al., 2020), as guided by (a) realist ontology that assumed an objective reality existed
independent of one’s perception of it (Wynn & Williams, 2012); (b) interpretivist epistemology
or the assumption that relationships, ideas, and knowledge interacted to shape causal
mechanisms, events, and experiences that resulted in specific tendencies (Peters et al., 2013); and
(c) the axiological goal to understand mechanisms and causal regularities (Bogna et al., 2020).
Toward the second study aim, the researchers explored structures, mechanisms, and conditions
associated with a well-established and empirically supported theoretical framework, BPNT, that
shaped these experiences using a critical realist paradigm.
The two paradigms were merged to explore the domain of the actual, where mutually
constructed meanings and associated occurrences and events were shaped by structures,
mechanisms, and conditions, and therefore, allowed a more complete and deeper understanding
of the studied phenomenon (Bogna et al., 2020; i.e., referral experiences). In other words, the
MENTAL HEALTH REFERRALS IN SPORT 9
multiparadigm approach allowed exploration of the what, using constructivism, and the how,
using critical realism, regarding mental health referrals in collegiate sport.
Methodological Approach
Consistent with a multiparadigm approach that aimed to explore the domain of the actual
(Bhaskar, 1978), a constructivist grounded theory methodology (Charmaz, 2000; 2006, 2014)
was integrated with a critical realist, or retroductive, grounded theory methodology (Looker et
al., 2021), both of which share eight common elements (Weed, 2009). According to Charmaz
(2014), a constructivist grounded theory methodology acknowledges “subjectivity and the
researcher’s involvement in the construction and interpretation of the data” (p. 14). Hence, this
methodology allowed for exploration of what are the referral experiences and what are the
realities of collegiate athletes within relevant culture and context, while considering the
researchers’ role, including experiences, privileges, and biases, in the construction of that
knowledge.
Rooted in classical grounded theory (Glaser and Strauss, 1967), retroductive grounded
theory aims to develop understanding of mechanisms, or causal powers, that produce the studied
phenomenon (Hoddy, 2019; Kempster & Parry, 2011; Looker et al., 2021). According to
Kempster and Parry (2011), because the critical realism paradigm suggests knowledge is valueladen, retroductive grounded theory is a context-rich, versus context-free, approach to studying
phenomena. Unique from other grounded theory approaches, drawing upon existing theories
while analyzing the data is encouraged with a retroductive approach (Kempster & Perry, 2011).
Hence, this retroductive methodology was used to explore the role of BPNT in clarifying the
possible mechanisms that affected mental health referral in sport; or in other words, the how
behind collegiate athletes’ referral experiences. Together, constructivist and retroductive
MENTAL HEALTH REFERRALS IN SPORT 10
grounded theory methodology captured the lived experiences and meanings of collegiate athletes
who were referred to mental health services, and mechanisms that shaped these experiences.
Participants
Participants were 14 current (n = 8) and former (n = 6) collegiate athletes aged 18 to 25
years (M = 21.28, SD = 2.23). Participants were referred one to three times within the past four
years by athletic trainers (n = 8), coaches (n = 7), collegiate athlete peers (n = 5), nutritionists (n
= 2), mental performance consultants (n = 2), and physicians (n = 1). Table 1 provides sample
demographics.
Table 1
Participants’ Self-Identified Demographic Characteristics
Characteristic n %
Sex and Gender
Cisgender Woman 10 71.4
Cisgender Man 4 28.6
Race and Ethnicity
*White 12 85.7
Latina 1 7.1
White and Asian 1 7.1
Nationality
American 11 78.6
English 1 7.1
German 1 7.1
Mexican 1 7.1
Annual Family Income
$25,000-$50,000 3 21.4
$50,000-$75,000 3 21.4
$75,000-$100,000 2 14.3
$100,000 or more 5 35.8
Did not know 1 7.1
Religion and Spirituality
Nondenominational Christian 3 21.4
Catholic Christian 3 21.4
Spiritual but not religious 2 14.3
Agnostic 1 7.1
Atheist 1 7.1
Easter Orthodox Christian 1 7.1
MENTAL HEALTH REFERRALS IN SPORT 11
Characteristic n %
Methodist Christian 1 7.1
Pentecostal/Assembly of God Christian 1 7.1
Protestant Christian 1 7.1
Sport
Rifle 2 14.3
Soccer 2 14.3
Swimming 2 14.3
Triathlon 2 14.3
Diving 1 7.1
Lightweight rowing 1 7.1
Rowing 1 7.1
Track and Field; Softball 1 7.1
Competitive Level
NCAA DI 10 71.4
NCAA DII 1 7.1
NCAA DIII 3 21.4
Note. NCAA = National Collegiate Athletic Association; DI/DII/DIII = Divisions I, II, and III
*5 participants who identified as white American also reported identifying as:
Bulgarian, German/Polish/Irish/Italian, Irish/German, Irish/Wales, and Polish.
Materials
Demographic Survey. A demographic survey (see Appendix E) was administered via
Qualtrics software and asked when participants were referred to mental health services and who
referred them, as well as their age, sex, gender identity, sexual orientation, race, ethnicity,
nationality, SES (socioeconomic status), religion, collegiate sport organization, division, and
sport(s).
Interviews. Individual in-depth, semi-structured, online, video-recorded interviews
started with rapport-building, followed by open-ended and follow-up questions to generate
conversation as recommended by Charmaz (2014) and Hoddy (2019). The first author, who
conducted the interviews, continuously engaged in self-reflexivity according to guidelines
established by Tracy (2010), to examine her assumptions about referrals in collegiate sport and
communicate, both verbally and via open body language, a willingness to learn and understand
MENTAL HEALTH REFERRALS IN SPORT 12
participants’ lives from their perspective. These experiences were processed with a critical friend
with each subsequent interview to also deepen reflexivity.
The first version of the interview guide was developed based on previous literature on
mental health referrals in sport (e.g., Sebbens et al., 2016; Van Raalte et al., 2015), BPNT (e.g.,
Ryan & Deci, 2017), prior experiences interviewing collegiate athletes about mental health
concerns (e.g., Coker-Cranney et al., 2018), as well as guidance on constructing questions
aligned with both constructivist (Charmaz, 2014) and critical realist aims (Brönnimann, 2022).
Broadly, questions explored collegiate athletes’ experiences being referred to mental health
services with intentional exploration of their identities and culture in connection with those
experiences. Participants were then provided a cursory overview of the three psychological needs
and asked their perceptions of needs satisfaction and thwarting amid their referral experiences.
Consistent with Glaser and Strauss (1967), the interview guide evolved as data were collected to
support the process of theoretical sampling and consequent development of grounded theory (see
Appendix F for the final version). Initial interviews ranged from 55 to 105 minutes in duration
(M = 86 minutes).
Procedures
After receiving IRB approval (Protocol #2302727613), participants were purposively
sampled according to the eligibility criteria (i.e., at least 18 years of age, current or former
collegiate athlete, was referred to mental health services in the past four years by support
personnel or other athletes). Maximum variation sampling was used to garner the widest range of
experiences, to the extent possible, with respect to several key factors found to play a role in
collegiate athletes’ mental health concerns and/or help seeking behaviors (e.g., Ballesteros &
Tran, 2020; Brown et al., 2021; NCAA, 2022; Li et al., 2017); these variables included sex,
MENTAL HEALTH REFERRALS IN SPORT 13
gender identity, sexual orientation, race and ethnicity, and SES. Further maximum variation was
sought with respect referral experience (i.e., positive and negative) and referral source (e.g.,
mental performance consultant, athletic trainer, sport dietician, academic advisor, strength and
conditioning coach, sport coach, or athlete peer). Nationality, sport, year in college, collegiate
sport organization, division, and religion were also considered.
As suggested by Charmaz (2014) and Hoddy (2019), interviews were transcribed and
evaluated as they were completed to support a theoretical sampling process that informed the
type of data sought in subsequent interviews as a theory emerged, such as revising interview
guide questions (e.g., re-ordering questions about basic psychological needs) and/or recruiting
participants with certain experiences (e.g., perceiving to have a lack of choice during referral
experience) or attributes (e.g., man-identified collegiate athletes) to explore a gap in the
developing theory. Recruitment, then, was an iterative process continuously informed by the
purposive, maximum variation, and theoretical sampling processes to obtain a sample with rich
and wide-ranging experiences that meaningfully informed a substantive grounded theory.
Recruitment approaches included multiple flyer (see Appendix C) distributions via social
media outlets (e.g., LinkedIn) and email to colleagues, sport support personnel (e.g., mental
performance consultants, athletic trainers), and or collegiate sport teams. Interested participants
who voluntarily contacted the first author were provided additional information about the study
outlined in a cover letter (see Appendix D), including the purpose of the study, what
participation would entail, the possible benefits and risks of participation, and participation
incentives, and then screened for inclusion criteria. Eligible participants were offered an
opportunity to read the cover letter, ask questions, and then complete a demographic survey. The
first author conducted initial interviews and invited the participant for a member reflection on a
MENTAL HEALTH REFERRALS IN SPORT 14
second date following the initial coding of the participant’s first interview. Although each
member reflection was unique, they broadly included a summary of main takeaways, followed
by a discussion of participant’s impressions and conclusions from the initial interview, and new
understandings and experiences since the initial interview. Thirteen participants completed the
follow-up, lasting between 10 and 29 minutes (M = 19.7). Participants who completed the study
received a $50 Amazon gift card in modest appreciation of their time. To protect participants’
confidentiality, all interviews were de-identified, and participants were assigned pseudonyms.
Data Analysis
Data analysis was guided by both constructivist (Charmaz, 2014) and retroductive
grounded theory approaches (Looker et al., 2021). Prior to and during data collection and
analysis, a two-person data analysis team continuously explored theoretical sensitivity –
examining the studied phenomenon with an open mind to explore and compare different
perspectives as well as evolve original ideas (Glaser, 1978). Theoretical sensitivity allowed the
data analysis team to explore the participants’ perspectives about referrals in novel and original
ways. To support this process, the data analysis team consulted with a larger research team,
comprised of applied practitioners and academic researchers in sport psychology and clinical
mental health, during bi-weekly hourly meetings for several months.
According to Glaser and Strauss (1967), grounded theory methodology involves an
iterative process of engaging in data collection and analysis simultaneously. Accordingly,
analysis started, and continued, as each new interview was conducted and transcribed. Analysis
of each interview began with the data analysis team reading each transcript and then verbally
processing their insights with one another. The first author then conducted the initial coding
(constructivist grounded theory) using a line-by-line coding approach to define the phenomenon
MENTAL HEALTH REFERRALS IN SPORT 15
and allow the data to speak (Charmaz, 2014). The goal of the initial coding was to remain open
and curious to all possible theoretical directions suggested by participants’ perceptions of the
referral process. Charmaz suggested focusing on exploration of the actions and processes that
define the phenomenon (i.e., collegiate athletes’ experience being referred to mental health
services) was crucial at the beginning stages of the data analysis because it helped provide
evidence of fit (i.e., capturing the essence of the participants’ experiences via codes and
categories) and relevance (i.e., creating a framework that captures what was happening and
relationships between implicit and explicit processes) of the emerging theory.
Then, the first author conducted focused coding (constructivist grounded theory), also
called open coding (retroductive grounded theory) – a process through which the goal was to
conceptualize the phenomenon by choosing the most significant and or frequent initial codes that
categorized the data inclusively and completely, and then comparing them to additional data
(Charmaz, 2014; Looker et al., 2021). During focused/open coding, the first author moved across
interviews to compare participants’ referral experiences, actions, and meanings (the dolman of
empirical) to synthesize the main themes across large amounts of data while also exploring the
structures and mechanisms that may have underscored these experiences and meanings (the
domain of real).
Theoretical categories (constructivist grounded theory) or emergent properties
(retroductive grounded theory; i.e., core elements in a theory) emerged during focused/open
coding and memo writing as well as constant comparison between the data (Charmaz, 2014;
Looker et al., 2021). This process was not linear; later interviews made some processes clearer
and incited reanalysis of earlier interviews. Following focused/open coding, the first author
engaged in theoretical coding – a process that hypothesized possible relationships among
MENTAL HEALTH REFERRALS IN SPORT 16
theoretical categories/emergent properties (constructivist grounded theory; Charmaz, 2014), with
retroductive coding that explored generative and context-specific causal mechanisms that helped
explain those relationships (retroductive grounded theory; Looker et al., 2021). Consistent with
constructivist and retroductive grounded theory, this phase involved constant comparison
between the new data, memos, and codes as well as possible reanalysis (Charmaz, 2014; Looker
et al., 2021) toward exploration of the domain of the actual.
To support the process of describing and conceptualizing the phenomenon, the first
author engaged in consistent memo writing described as an analytic process in both
constructivist and retroductive grounded theory (Charmaz, 2014; Looker et al., 2021). Broadly,
memo writing allowed the first author to generate ideas, develop and connect theoretical
categories/emergent properties, gain insights early in the research process, and explore the
possible causal mechanisms that explain the relationships between theoretical
categories/emergent properties (Charmaz, 2006; 2014; Looker et al., 2021). Memo writing
supported iterative data analysis as it allowed the first author to explore and discover ideas freely
and spontaneously (Weed, 2009). When needed, memos were discussed with the research team
to support the exploration of the phenomenon.
An important part of memo writing was raising focused/open codes to theoretical
categories/emergent properties (Charmaz, 2014; Hoddy, 2019). This was done by choosing
focused/open codes that best capture what was happening in the data and why it was happening
though comparison between the data, incidents, contexts, and other categories/properties. Via
memo writing, the first author defined and explored characteristics of tentative theoretical
categories/emergent properties, explored their relationships with other theoretical
categories/emergent properties, and possible causal mechanisms that explained these
MENTAL HEALTH REFERRALS IN SPORT 17
relationships (Charmaz, 2014; Looker et al., 2021). In advanced memos, the first author traced
and categorized data to achieve the purpose of the study, describe how theoretical
categories/emergent properties and possible causal mechanisms emerge and change, identify the
beliefs and assumptions that supported them, explore various perspectives of the memo topic,
and engage in constant comparisons. Throughout each phase of the study, the first author
conducted constant comparison iteratively between data and data, data and codes, codes and
theoretical categories/emergent properties, and theoretical categories/emergent properties and
BPNT literature.
The first author started theoretical sampling once there were preliminary theoretical
categories/emergent properties from which to collect subsequent data that expanded and
improved theoretical categories/emergent properties and related causal mechanisms informing
the emerging theory (Weed, 2009). Theoretical sampling was facilitated by memo writing that
helped identify incomplete theoretical categories/emergent properties and causal mechanisms,
and gaps in the analysis. Weed (2009) argued theoretical sampling, based on tentative theoretical
categories/emergent properties, continued until no new characteristics of theoretical
categories/emergent properties and no new causal mechanisms emerged from new data (i.e.,
theoretical saturation; Charmaz, 2014; Hoddy, 2019). Based on what emerged from the existing
data, theoretical sampling involved recruiting new participants with specific experiences, or
collecting follow-up data collection from the existing participants (Charmaz, 2014; Hoddy,
2019). The first author regularly consulted with the research team about decisions regarding the
specific approaches to theoretical sampling.
To deepen the analysis and construction of the emerging theory, the first author engaged
in theoretical sorting, comparing/diagramming, and integrating memos (Charmaz, 2014). This
MENTAL HEALTH REFERRALS IN SPORT 18
involved comparing memos and exploring how the order and logic of memos capture the referral
experiences of collegiate athletes. Additionally, drawing diagrams assisted in establishing and
solidifying theoretical categories/emergent properties, their relationships, and causal mechanisms
that explain them. Together, these strategies provided insight toward an emerging theoretical
framework on referring collegiate athletes to mental health services. To support the process of
theory development, the first author met with expert consultants regularly during each phase of
the analysis regarding the quality and rigor of the analysis and relevance to the BPNT
framework. Follow up member reflections and the research team interpretations were transcribed
and analyzed with the rest of the data to refine the theoretical categories/emergent properties and
emerging theory (Albas & Albas, 1988; 1993; Charmaz, 2014; Looker et al., 2021).
Methodological Rigor
To support the process of investigating, interrogating, and deepening different
interpretations of the data (Harper, 2011), the research team included individuals with a variety
of experiences in the collegiate sport and mental health fields. Some of the shared experiences of
the research team involved completing post-secondary education in either sport psychology,
mental health counseling, or both. Additional commonalities among the research team included
being an athlete, a coach, a mental performance consultant, and a mental health counselor in
various sport and clinical settings in the United States and Europe. Some of the unique
experiences of the research team included identifying with different nationalities (i.e., American,
German, Austrian, and Serbian) and gender identities (i.e., woman- and man-identified),
competing and working for different sports and sport levels (i.e., high school and collegiate), and
interacting with collegiate athletes in different capacities (e.g., as a therapist, a professor, a
mental performance consultant, a supervisor, a former collegiate athlete). Additionally, the
MENTAL HEALTH REFERRALS IN SPORT 19
research team had diverse experiences, expertise, and formal training regarding conducting
qualitative research in sport psychology and counseling, BPNT, and making and receiving
collegiate athletes’ mental health referrals.
The approaches used to support data trustworthiness, credibility, and confirmability were
engaging in self-reflexivity both in written (i.e., memo writing) and verbal (i.e., discussions with
the critical friend) forms as well as regular consultations with the research team experts
regarding conceptual and methodological decisions, and conducting member reflections, as
previously discussed. Additionally, Carcary (2020) recommendations for audit trail best practices
were followed (i.e., documenting the research process in all phases of the study, including key
methodological decisions, and tracing how researchers’ thinking evolved during each phase).
Results
Data analysis resulted in a substantive grounded theory (Figure 1) that described and
explained participants’ experiences of being referred to mental health services by support
personnel and other athletes. Three overarching theoretical categories/properties emerged: (a)
pre-referral factors (i.e., conditions and dynamics prior to referrals informing referral
conversations); (b) referral conversations (i.e., interactions in which referrers recommended
athletes seek appropriate mental health resources); and (c) referral outcomes (i.e., consequences
of referral conversations). Figure 1 illustrates subcategories identified within each category, and
the relationships between them, that together described participants’ experiences and
mechanisms explaining those experiences.
Figure 1
A Substantive Grounded Theory of Referring Collegiate Athletes to Mental Health Services
MENTAL HEALTH REFERRALS IN SPORT 20
Note. White boxes represent experiences, and gray boxes represent mechanisms.
Pre-Referral Factors
Prior to the initiation of referrals, pre-referral factors shaped how referral conversations
unfolded and, ultimately, the referral outcomes. These factors included aspects of participants’
identities and cultures and interactions with their family and sport environments. Although basic
psychological needs were not explicitly explored with participants at this stage of the interview,
pre-referral factors clearly also influenced autonomy, competence, and relatedness when shaping
participants’ relationships with, and attitudes toward, mental health and, ultimately, the referral.
Participants’ Identities and Culture
Participants described how aspects of their identities and culture, such as their nationality,
race, ethnicity, SES, religion, and gender identity, influenced their perceptions of mental health
concerns and help-seeking behaviors that, in turn, informed their willingness to pursue referral
MENTAL HEALTH REFERRALS IN SPORT 21
recommendations. For example, Fin referenced that “in Germany, we are…more private, very
distant people” which is “probably also one of the reasons why I never really looked at [therapy]
before.” Mia, on a different note, commented on SES status: “Growing up, the money aspect was
hard. So, that’s why I never went [to therapy].” Other participants described how binary gender
norms, informed by their upbringing and traditional sport culture, influenced their help-seeking
attitudes. Gia reported that “girls are seen as weaker…guys [are] trying to be the strongest” and
that “coaches would see you as weaker” for seeking mental health resources. Tom described how
a lack of representation contributed to his mental health concerns and deterred him from seeking
help: “I think not only is it hard to be a queer person [in the] outside world, but also being a
queer person inside of athletics is incredibly difficult.”
Further, some participants cited their religious background and spirituality as both a
barrier and facilitator to seeking mental health services. Bri explained, “[My religious
community is] very supportive…It gave me an opportunity…to talk about [my mental health]
more.” Zoe similarly discussed how “the guiding principles of religion” encouraged her to “find
the healthiest version of myself, mentally and physically.” Ana stated why she went to therapy:
I realized there was a spiritual element to this… The enemy was telling me, ‘You’re a
failure.’ And I was believing it… I need to get my faith back in order. This is a lie… This
is not who God says I am… That faith…was HUGE because that was the catalyst that
started changing things.
Other participants believed religion sufficed as therapy and, accordingly, avoided facing their
difficulties as a way to cope (i.e., spiritual bypassing). Mia reported, “I always just thought I
could just pray it off…Why isn’t praying enough for me? I didn’t want it to feel like I couldn’t
do it on my own with just God.”
MENTAL HEALTH REFERRALS IN SPORT 22
Some participants reported ways their intersecting identities shaped their views on helpseeking. Ana described her “quite emotionally unavailable” English culture and “the culture of
being an athlete” as making her “incredibly reluctant” to seek help. Others explained that support
personnel failed to honor their intersecting identities, exacerbating their mental health concerns
and unwillingness to seek help. Tom explained that, in his sport environment, the term “athlete”
was used to generalize “a group of people that can be incredibly diverse.” He noted, “As
someone who has been athletic since I was five years old, I just took onto that [athlete] title and
didn’t allow other things in my life to define who I was.” He added, “[Sport personnel] don’t
really talk too much about sexuality…It was never really a topic of conversation.”
For the participants who recognized having privileged identities, representation in mental
health settings was often not a concern. Ana stated, “I’m heterosexual, [that is] why I don’t feel
there [were] any barriers in terms of those parts of my identity.” Ana also acknowledged her
teammates may have had a different experience: “We had a couple of lesbians on the team. I
could see how they wouldn’t want to go to a Christian counselor for fear that they would be
judged.” For some participants who belonged to minoritized groups, representation was key to
their willingness to seek mental health services. As a member of “the international community,”
Lea reported noticing “more of a need for cross-cultural specialists… Our counseling center…
has been staffed by older white people.” Lea suggested “having…knowledge of various
backgrounds and…cultures…helps with the referral process” as professionals may “understand
your experience and you feel represented in them.” Participants’ diverse identities and culture
had strong effects on their help-seeking attitudes and behaviors, and, consequently, their
openness to referral recommendations.
Participants’ Family Environments
MENTAL HEALTH REFERRALS IN SPORT 23
Participants in family environments that encouraged help-seeking were more likely to
have positive attitudes toward help-seeking themselves. Max stated, “I’ve never thought [seeking
therapy] was a bad thing… I’ve always been told…if you’re struggling mentally, there’s nothing
wrong with talking to [psychotherapists].” Ana noted, “My parents didn’t see seeking counseling
as a sign of weakness… I know my dad had gone…and…it was helpful for him… If anything, it
encouraged me… They’re not gonna judge me…” Contrarily, negative attitudes toward help
seeking in their immediate family deterred other participants from wanting to seek mental health
services. Mia mentioned, “My parents are very against being mentally weak… [My extended
family] don’t believe in therapy at all… I would never say [to them]…that I’m in therapy.”
Some participants also said that help-seeking attitudes were passed down generationally,
making it harder to shift their own attitudes. Lea reported, “[My mother] moved from [another
country]… With my grandparents it very much was like, ‘You don’t need anything. It’ll be
fine’…The cultural mindset travels with you.” Among these participants, some discussed slow
but important shifts in intergenerational help-seeking attitudes. Gia reported that her “transition
generation” is growing up in a time when they are “actually…listened to” instead of being told to
just “toughen up,” and that younger generations will benefit from further shifts in this direction.
Family members’ attitudes towards mental health help-seeking, whether positive or negative,
strongly influenced participants’ own attitudes yet were noted as malleable.
Participants’ Sport Environments
Many participants similarly described their sport environments as supportive or
unsupportive of mental health help. These participants were more likely to have positive attitudes
toward help-seeking when athletic department cultures encouraged it. Joy felt “really good”
about seeking services after noticing that “no one looked twice” when her teammate disclosed
MENTAL HEALTH REFERRALS IN SPORT 24
pursuing therapy. Gia did not originally value mental health services, but because her head coach
“wants to talk about mental health…it’s not uncomfortable.” Gia also described several benefits
of mental health services being accessible and frequently promoted in her sport environment. Per
Gia, coaches pointed to “posters with [the psychotherapist’s] contact information,” and athletic
trainers frequently offered to “help you set up an appointment.” These participants also discussed
having better access to services when mental health providers were integrated into sport. For
Max, “the fact that [the psychotherapist in athletics] is around us and really looks out for
us…definitely helped. Max noted, “I don’t know if it was some random person if I would have
[sought therapy].”
Conversely, some participants noted several barriers when athletic department cultures
failed to promote mental health. Ana’s teammate “didn’t believe what was going on with me was
a big deal… It made me feel like, ‘Oh, maybe she’s right. Maybe there’s nothing wrong with me.
Maybe I’m just being dramatic,’ which was very painful.” Ana further noted, “Before that
conversation [about referrals] even happens, …there’s more that can be done with the culture
around [mental health],” explaining that requiring a referring athletic trainer to “deal with this” is
“unfair…when there is not a culture already in place that makes those conversations accessible
and comfortable.” Fin “wasn’t really aware of [existing therapy resources for athletes]… that
wasn’t really too public that I would have the option to talk to [psychotherapists]…. That
message never really came from the main coaches.” Ella had “never heard someone who goes to
[the psychotherapist in athletics] and has a good experience… She’s trained…just…not very
good.” Contrarily, some participants indicated that an overemphasis on mental health was also an
issue. Ella described this as an “overcorrection,” as her coach’s efforts to normalize mental
health made the coach “too emotionally involved”, thus “hindering…[team] performance.” Ella
MENTAL HEALTH REFERRALS IN SPORT 25
also felt that some of her teammates were “too annoying and loud about mental health.” As with
family environments, participants’ sport environments, including cultural messaging about
mental health, impacted their pre-referral attitudes and intentions toward help-seeking.
Reactive and Proactive Referral Conversations
Participants’ identities, cultural backgrounds, and family and sport environments
informed the help-seeking attitudes they, figuratively, brought with them to their referral
conversations with support personnel and other collegiate athletes. In these referral
conversations, referrers broached their concerns with participants and recommended mental
health resources or services. Many participants described these conversations as having occurred
reactively (i.e., in response to an experienced mental health crisis) or proactively (i.e., with the
aim to prevent concerns from beginning or worsening). Participants further discussed ways they
perceived their basic psychological needs to be supported or thwarted amid these interpersonal
exchanges.
Most participants reported being referred reactively, as a result of “hitting rock bottom”
(Tom) or in response to crisis, such as sexual assault, suicidality, and panic. Referrals following
a crisis that were experienced positively involved the referrers who promptly provided support,
information, and choices to help them feel safer and more in control. Liz stated it “was helpful to
know what I was walking into.” She further explained, “[The athletic trainer] said, ‘[The
boyfriend]…can sit in [the session] with you…,’ prepared me it was going to be a guy [the
psychotherapist],” and gave “me the basics of what I was going to experience.” In rare cases,
some participants were referred proactively as a preventive measure to help them cope with
upcoming stressors. Gia, who “had never done triathlon before”, explained that her head coach
referred her “to make sure that you’re not feeling you’re in over your head.” Being referred
MENTAL HEALTH REFERRALS IN SPORT 26
proactively increased some participants’ openness to services and reduced mental health stigma.
For Gia, this approach showed her that “anyone can be referred… [Therapy] can be helpful for
anybody” and “preventative.” Lea described being referred proactively as “a welcome little push
to get me to take care of myself and prioritize wellbeing,” with others stating that it helped them
overcome ambivalence or avoidance. Bri noted, “Had I not had that [referral] conversation with
[the athletic trainer], I probably would have never gone to see anybody.” Such examples
exemplified the importance of approaching each referral differently based on an athlete’s unique
needs.
Psychological Need Satisfaction and Thwarting
In addition to the proactive/reactive nature of referral conversations, participants
discussed ways in which the interpersonal exchanges with referrers supported, or thwarted, their
basic psychological needs. These included participants’ autonomy (i.e., sense of selfgovernance), relatedness (i.e., sense of connection), and competence (i.e., sense of mastery).
Autonomy Satisfaction and Thwarting. Many participants explained that common
hierarchical power structures influenced their referral conversations and, subsequently, their
decisions to pursue mental health services. When asked about what it was like to be referred by
their teammates, two participants voluntarily compared it to referrals by coaches, stating it felt
“more like my own decision… Athlete to coach is more a sense of authority” (Eva). Mia echoed,
“If [the coach] were to suggest [therapy]… I would not have a comfortable conversation” and
“would just go…feel I had no choice” and “be doing it for him.” For Ari, “Having [the athletic
trainer and the coach]…reach out to [the psychotherapist] before I even gave consent…or
approval…was upsetting. So, I didn’t feel I had the autonomy because it was not brought
forward to me.” In contrast, a few participants discussed how financial privilege was another
MENTAL HEALTH REFERRALS IN SPORT 27
form of power that supported autonomy during and following the referral. Bri reported she “went
out and found [a psychotherapist] on my own… It’s privileged to be able to afford that…
Privilege does create autonomy.”
Many participants reported feeling autonomous when their referrers asked them for their
thoughts and preferences and provided control around referral outcomes, including the
opportunity to contemplate and choose whether to pursue services. Bri stated, “it was my choice”
because the athletic trainer informed her about resources and said “It’s completely up to you.
We’re not going to force you into it or judge you if you decide not to.” Fin reported, “[Therapy]
was…brought up…[as] an option. And I was like, ‘Okay, I’m gonna give it a thought…’ It was
never the end of the conversation and then a fixed, ‘Yeah, I’m gonna…[seek therapy].’” Fin
further explained, “After a week… based on m
Les avis et les commentaires de MistralAI
From Measur om Measurement t ement to Performance: Psy formance: Psychometric V chometric Validity and alidity : From Measur om Measurement t ement to Performance: Psy formance: Psychometric V chometric Validity and alidity and
Psychological Pr chological Predictors of Reser ors of Reserve Officer T e Officer Training Corps aining Corps
Cadets’ Performance on the Army Combat Fitness Test
Ari Joseph Sapinsley
West Virginia University
Follow this and additional works at: https://researchrepository.wvu.edu/etd
Part of the Sports Studies Commons
Recommended Citation
Sapinsley, Ari Joseph, "From Measurement to Performance: Psychometric Validity and Psychological
Predictors of Reserve Officer Training Corps Cadets’ Performance on the Army Combat Fitness Test"
(2024). Graduate Theses, Dissertations, and Problem Reports. 12672.
https://researchrepository.wvu.edu/etd/12672
This Dissertation is protected by copyright and/or related rights. It has been brought to you by the The Research
Repository @ WVU with permission from the rights-holder(s). You are free to use this Dissertation in any way that is
permitted by the copyright and related rights legislation that applies to your use. For other uses you must obtain
permission from the rights-holder(s) directly, unless additional rights are indicated by a Creative Commons license
in the record and/ or on the work itself. This Dissertation has been accepted for inclusion in WVU Graduate Theses,
Dissertations, and Problem Reports collection by an authorized administrator of The Research Repository @ WVU.
For more information, please contact researchrepository@mail.wvu.edu.
From Measurement to Performance: Psychometric Validity and Psychological Predictors of
Reserve Officer Training Corps Cadets’ Performance on the Army Combat Fitness Test
Ari J. Sapinsley, MA, MS, CMPC
A Dissertation submitted
to the College of Applied Human Sciences
at West Virginia University
in partial fulfillment of the requirement for the degree of
Doctor of Philosophy in
Sport, Exercise, and Performance Psychology
Sam Zizzi, Ed.D., Co-Chair
Johannes Raabe, Ph.D, CMPC, Co-Chair
Jeanette Garcia, Ph.D.
William Hornsby, Ph.D.
Department of Sport, Exercise, and Performance Psychology
Morgantown, West Virginia
December 13, 2024
Keywords: Psychometric evaluation, ROTC cadets, ACFT (Army Combat Fitness Test),
Psychological predictors, Motivation, Resilience, Anxiety
Copyright 2024 Ari J. Sapinsley
Abstract
From Measurement to Performance: Psychometric Validity and Psychological Predictors of
Reserve Officer Training Corps Cadets’ Performance on the Army Combat Fitness Test
Ari J. Sapinsley, MA, MS, CMPC
Understanding the psychological and physical readiness of future military leaders is
crucial for improving recruitment and retention. The U.S. Army faces increasing obesity rates
and declining physical fitness among recruits, reducing the pool of eligible candidates and
raising costs. The Reserve Officer Training Corps (ROTC), which commissions 59% of officers,
has higher attrition rates than other sources, underscoring the need to identify psychological
predictors of performance. This dissertation examines the psychometric properties of
psychological instruments used to assess ROTC cadets and their relationship with Army Combat
Fitness Test (ACFT) performance. Two research questions were explored: (1) What are the
psychometric properties of instruments measuring the psychological characteristics of ROTC
cadets? (2) What psychological characteristics are associated with ACFT performance? A
sample of 180 cadets completed assessments, including the Academic Motivation Scale –
Revised (AMS-R), Academic Motivation Scale – College (AMS-C), Basic Psychological Need
Satisfaction and Frustration Scale (BPNSFS), Cognitive and Somatic Anxiety Inventory – 2
Revised (CSAI-2R), and Resilience Scale (RS). Confirmatory factor analyses (CFA) indicated
acceptable reliability and validity for most instruments, with strong model fit for CSAI-2R (CFI
= 1.00, TLI = 0.99, α = 0.94) and AMS-C (CFI = 0.95, TLI = 0.95, α = 0.94). Regression
analyses identified resilience (β = 0.21, p < 0.05) as a significant predictor of hand-release pushup scores, while self-confidence (β = 0.33, p < 0.05) predicted two-mile run performance.
Cognitive anxiety was positively associated with the two-mile run (β = 0.31, p < 0.05),
suggesting moderate anxiety may enhance focus. Academic identified regulation and intrinsic
motivation were also significant predictors. This study provides insights into the psychometric
properties of thirteen instruments among ROTC cadets and elucidates how psychological traits
affect ROTC performance. These findings support integrating psychological assessments into
training and selection processes to improve cadet identification, training, retention, and cost
efficiency. Future research should validate these results in larger, diverse samples and use
longitudinal designs to explore long-term impacts.
iii
Acknowledgments
Although, technically, this is my dissertation, this document and project only exist
because of a village of wonderful and supportive family, friends, colleagues, and mentors. First
and foremost, I would like to thank Dr. Raabe. Your work ethic, perspective, and guidance have
been and will continue to be pivotal to my development as a researcher and writer. I am
incredibly grateful for your support and commitment to my (our) work, and the skills you have
helped me hone over the past three and a half years.
To Dr. Zizzi – thank you for stepping in when you, most certainly, had enough on your
plate, making me feel comfortable and confident as a student and researcher, and for your
limitless supply of pragmatism. If not for you, I would have never seen the forest through the
trees or been able to traverse Research Mountain. I will forever be grateful for the impact you
have had on me during my time in the Sport, Exercise, and Performance Psychology program at
WVU, as well as for your continued contributions to our program and the broader field.
To Dr. Shigeno and Dr. Fletcher – thank you both for your patience and your open doors.
When I thought, “I think I want to pursue even more education,” you both met me with
excitement and encouragement. Under your tutelage, I began my journey in the field of sport
psychology, and you both have continued to be voices of reason and mentorship long after my
departure.
To my parents – thank you for providing me with the privilege and opportunity to pursue
any dream, no matter how far-fetched it may have seemed. Your unwavering support,
unconditional love, and steadfast belief in me have been the bedrock of everything I have
accomplished. You taught me the value of hard work, resilience, and the courage to forge my
own path, even when that path was unclear or daunting. I am endlessly grateful for your
encouragement, your wisdom, and the example of love and perseverance you’ve set for me.
To my brothers – Aaron, Zack, and Josh, thank you for your friendship (although mostly
involuntary). From you all, I have learned about debate, intellectual curiosity, competition,
friendship, and cooperation. You have each shaped who I am in your own unique way,
challenging me to think critically, encouraging me to strive for excellence, and teaching me the
value of both standing my ground and finding common ground. Our shared experiences, whether
marked by laughter, spirited disagreements, or quiet support, have been a source of strength and
inspiration throughout my life. I am grateful for the bonds we share and the lessons I continue to
learn from you. Thank you for always being in my corner, even when we didn’t see eye to eye—
it has made the journey all the more meaningful.
To my in-laws – Mike and Peggy, thank you for your endless supply of generosity, for
opening your home to me, and for always having a koozie at the ready. Your unwavering support
and kindness have had a profound impact on my educational journey. From the very beginning,
you welcomed me into your family with open arms, creating a space where I felt cared for and
supported. Your constant encouragement, whether through thoughtful conversations, shared
laughter, or the take-home supply of minestrone, has been invaluable. Thank you for being such
a vital part of this journey, your impact on my life and this accomplishment is something I will
always treasure.
To my wife – thank you, Mary, for your unwavering support, love, and friendship. You
have been my rock, my cheerleader, and my constant through every stage of this journey, from
undergrad to my master's and now my PhD. Together, we have navigated the highs and lows,
celebrated victories, and weathered the challenges that came with this long and winding path.
iv
Your patience, encouragement, and belief in me, even on the days when I doubted myself, have
been nothing short of extraordinary. You’ve sacrificed so much to support my goals, and your
faith in my ability to succeed has never faltered. I am endlessly grateful for your partnership and
the countless ways you’ve made this journey not only possible but meaningful. This
accomplishment is ours, and I am so lucky to have you by my side, not just in this, but in all
things.
Lastly – thank you to Lyla, our dog, you are, without a doubt, a good girl. When I was
stressed, you were always available for a pet and a cuddle. When I was huddled over my
computer for long stretches, you were always there to remind me that we should go outside for a
walk. Your resolute loyalty and cheerful spirit brought light to even the most challenging days,
and your gentle nudges reminded me to pause, breathe, and appreciate the world beyond my
work. You’ve been my constant companion, my motivator, and my joy through this journey, and
for that, I’m endlessly grateful.
v
Table of Contents
From Measurement to Performance: Psychometric Validity and Psychological Predictors of
Reserve Officer Training Corps Cadets’ Performance on the Army Combat Fitness Test............ 1
Method ............................................................................................................................................ 9
Participants.................................................................................................................................. 9
Study Design............................................................................................................................... 9
Measures................................................................................................................................... 10
Data Analysis............................................................................................................................ 17
Results........................................................................................................................................... 19
Psychometric Properties of Psychological Instruments (RQ1)................................................. 19
Psychological Predictors of ACFT Performance Among ROTC Cadets (RQ2)...................... 20
Discussion..................................................................................................................................... 21
Psychometric Properties of Psychological Instruments (RQ1)................................................. 21
Psychological Predictors of ACFT Performance Among ROTC Cadets (RQ2)...................... 31
Limitations and Future Directions................................................................................................ 37
Summary and Conclusion............................................................................................................. 39
References..................................................................................................................................... 41
Tables............................................................................................................................................ 58
Extended Review of the Literature ............................................................................................... 89
Military ..................................................................................................................................... 89
Psychosocial Factors............................................................................................................. 90
Psychosocial Predictors of Physical Performance and/or Selection ................................... 106
Sport........................................................................................................................................ 123
Psychosocial Factors........................................................................................................... 128
Psychosocial Predictors of Physical Performance and/or Selection ................................... 137
Future Research Directions..................................................................................................... 148
Extended References................................................................................................................... 151
Appendices.................................................................................................................................. 169
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 1
From Measurement to Performance: Psychometric Validity and Psychological Predictors
of Reserve Officer Training Corps Cadets’ Performance on the Army Combat Fitness Test
Talent development within the United States (U.S.) Army is essential for maintaining
operational readiness, ensuring mission success, and fostering organizational adaptability. Army
officers must possess a blend of physical endurance, psychological resilience, and cognitive
adaptability to navigate the multidimensional demands of military service (U.S. Department of
the Army, 2023). However, challenges in officer recruitment, selection, and retention persist,
threatening the sustainability of the Army’s leadership pipeline. Only approximately 23% of
Americans aged 17 to 25 meet the eligibility requirements for military service, further
emphasizing the importance of retaining already trained personnel to sustain the force’s
effectiveness (Office of People Analytics, 2023). In response, the Army has renewed its focus on
prioritizing quality over quantity in officer development, shifting toward a holistic approach to
recruitment, training, and retention. Understanding the factors that predict successful officer
performance, as well as why officers choose to remain or exit after fulfilling their Active Duty
Service Obligation (ADSO), has become a burgeoning area of research aimed at improving
resource allocation and organizational maintenance (Russell & Tremble, 2011).
To address these challenges, the Army has emphasized integrating physical, cognitive,
and psychological measures into its assessment systems. This holistic approach seeks to ensure
the selection and development of officers capable of meeting evolving operational requirements
(Allen et al., 2016). For example, standardized assessments such as the Tailored Adaptive
Personality Assessment System (TAPAS) have undergone extensive psychometric evaluation to
improve the precision of personnel selection and position assignments. These tools have
provided valuable insights into factors that facilitate or inhibit organizational commitment
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 2
among active-duty soldiers (Kirkendall et al., 2020; Ellis et al., 2023). However, while efforts to
evaluate and refine psychological assessment tools have yielded meaningful advances for activeduty personnel, little attention has been devoted to understanding psychological predictors of
performance within the Reserve Officer Training Corps (ROTC), the Army’s largest source of
commissioned officers.
The Role of ROTC
The ROTC is a college-based program that integrates military training with higher
education, preparing students to become commissioned officers in the Active Duty Army, Army
Reserve, or Army National Guard. Approximately 60% of newly commissioned Army officers
come from ROTC programs, making it a cornerstone of the Army’s leadership pipeline (Paullin
et al., 2014). Cadets participating in ROTC take military science courses covering leadership
principles, team dynamics, physical readiness, and military tactics (U.S. Department of the
Army, 2023). Beyond classroom instruction, cadets engage in field training exercises, leadership
development activities, and physical fitness programs designed to foster the skills and
competencies necessary for future leadership roles. Upon completing the program and earning
their degrees, cadets are commissioned as Second Lieutenants, assuming critical leadership
responsibilities within the Army.
Additionally, cadets participate in summer training programs like Advanced Camp, a
rigorous evaluation of leadership potential and tactical proficiency held at Fort Knox, Kentucky
(U.S. Army Cadet Command, n.d.). These experiences, alongside regular ROTC training
activities, immerse cadets in military culture, teamwork, and the demands of active-duty service.
While invaluable for preparing cadets for their future roles, these requirements highlight the dual
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 3
pressures of balancing civilian life with military expectations, a challenge unique to the ROTC
pathway compared to academy cadets or enlisted personnel (Baldwin & Young, 2020).
Despite its importance, ROTC remains underexplored in research on military
performance and retention. Compared to their counterparts at U.S. Military Academies, ROTC
cadets operate in a dual-context environment that combines civilian academic responsibilities
with military training. While this structure allows cadets to benefit from a less regimented
experience and broader social interactions, it also introduces unique stressors. Cadets must
balance the demands of academic coursework, leadership roles, and physical fitness
requirements, all of which contribute to their overall readiness and success (Baldwin & Young,
2020; Legree et al., 2014). These challenges are compounded by ROTC’s more flexible
structure, which contrasts sharply with the fully militarized environments of academies like West
Point. Understanding the specific factors influencing ROTC cadet performance and retention is
critical for sustaining the Army’s leadership pipeline and ensuring the readiness of future
officers.
Assessment of ROTC Cadets’ Performance
The evaluation of ROTC cadets encompasses several performance domains, including
academic achievement, leadership capability, and physical readiness. Key metrics include Grade
Point Average (GPA), evaluations during the Leadership Development and Assessment Course
(LDAC), cadre ratings of leadership potential, and scores on the Army Combat Fitness Test
(ACFT). Together, these metrics contribute to the Order of Merit List (OML), which determines
cadets’ branch assignments and positions upon commissioning (U.S. Army Cadet Command,
n.d.; Baldwin & Young, 2020). Among these, the ACFT—introduced in 2020 to replace the
Army Physical Fitness Test—plays a pivotal role in assessing physical readiness. Comprising six
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 4
events, including the deadlift, standing power throw, hand-release push-ups, sprint-drag-carry,
plank, and a two-mile run, the ACFT provides a comprehensive measure of physical fitness (U.S.
Department of the Army, 2023). While the ACFT offers valuable insights into cadets’ physical
capabilities, it fails to account for the psychological factors that influence performance.
Improving ACFT performance is vital not only for individual cadets’ success but also for
broader Army organizational readiness. Cadets who fail to achieve high ACFT scores risk failure
and attrition in their ROTC program along with stagnation in their military careers (Newman et
al., 2022). These individual outcomes have cascading effects on leadership development and
retention within the Army. For instance, as the Army faces a drastically reduced pool of eligible
recruits, with only 23% of Americans between 17 to 25 years of age meeting military eligibility
criteria (Office of People Analytics, 2023), reducing attrition of eligible ROTC cadets and
retaining already trained personnel is paramount to organizational maintenance and continuity
(Wardynski et al., 2010; White et al., 2017). Therefore, it is financially and institutionally
essential to understand how to improve ROTC cadets’ physical readiness and ACFT capabilities.
Psychosocial Predictors of Military Performance
Maximizing ACFT performance requires a comprehensive approach that considers not
only physical capabilities but also the psychological factors that underpin consistency and
adaptability in high-pressure environments (U.S. Department of the Army, 2023). While physical
attributes like aerobic capacity, muscular strength, and agility are well-documented performance
predictors in fitness tests and occupational tasks (Acevedo et al., 2024; Hauschild et al., 2016;
Hydren et al., 2017; Orr et al., 2021), psychological readiness plays an equally critical role in
military performance and career longevity (Flood & Keegan, 2022). Initiatives like the
Comprehensive Soldier and Family Fitness program underscore the military's commitment to
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 5
psychological readiness for long-term career success (Casey, 2011). Maintaining mental wellbeing has also been found to reduce attrition and improve operational effectiveness among
active-duty and reserve soldiers (Tidwell & Lucier-Greer, 2024), underscoring that mental
preparation is essential for sustained military effectiveness. Emerging research further
emphasizes the importance of psychological factors in predicting performance, especially under
stress, to boost readiness and retention (Richardson, 2023).
The literature exploring psychological predictors of military performance has identified
numerous traits associated with success. Key psychosocial factors – emotion regulation (Giles et
al., 2023), psychological skills (Arthur et al., 2017; Hammermeister et al., 2010), mental
toughness (Gucciardi et al., 2021), and hardiness (Normdo et al., 2022) – appear to correlate with
military performance and selection. Longitudinal investigations of military personnel
performance and selection in rigorous training courses have also highlighted predictive
psychological characteristics such as grit (Benedict et al., 2023; Farina et al., 2019; Kelly et al.,
2014; Schaefer et al., 2020a), resilience (Farina et al., 2019; Schaefer et al., 2020a), hardiness
(Johnsen et al., 2013; Kelly et al., 2014), confidence (Schaefer et al., 2020b), optimism (Schaefer
et al., 2020b), self-efficacy (Benedict et al., 2023), and coping (Johnsen et al., 2013).
Among these studies, Giles et al. (2023) have been the sole research endeavor to
incorporate the Army Combat Fitness Test (ACFT) as an outcome variable. This study employed
psychometric network analysis on a sample of male active-duty US Army personnel (N = 191,
Mage = 23.1, SD = 3.5) to examine cognitive, health, physical, and social-emotional factors
associated with ACFT performance. Giles and colleagues (2023) found that social-emotional
components, such as resilience, extraversion, and grit, significantly explained performance
variance with minimal redundancy. However, the author’s focus on active-duty male soldiers and
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 6
the absence of psychometric validation limit the applicability of their findings to ROTC cadets,
whose training context and developmental stage differ significantly (Baldwin & Young, 2020).
Despite these limitations, such findings are promising and potentially foretell psychological
factors as significant predictors of ROTC students’ performance on the ACFT.
Although prior studies provide valuable insight into psychological predictors of military
performance, research exploring military populations still faces several limitations. First, many
studies are constrained by the relative homogeneity of their samples (e.g., Benedict et al., 2023).
Second, while military academy studies (e.g., Kelly et al., 2014; Schaefer et al., 2020a) may
display improved heterogeneity, they typically involve participants who meet higher academic
and physical performance standards compared to traditional ROTC cadets (The United States
Military Academy, n.d.). Third, much of the literature has been conducted outside the US (e.g.,
Bekesiene, 2023; Normdo et al., 2022), limiting the generalizability of findings to U.S. military
settings due to significant differences in structure, requirements, and education. Lastly, many
studies suffer from a lack of female representation in their samples, with some studies including
no female participants at all (e.g., Hormeño-Holgado et al., 2019). This underrepresentation fails
to reflect the makeup of the U.S. Army, which comprises 15.6% women (U.S. Department of the
Army, 2022), further limiting the applicability of the research.
Validity and Reliability of Psychological Instruments for ROTC Cadets
To address the research gaps, the current study aimed to evaluate psychological
constructs crucial for military performance, particularly in the context of ROTC cadets.
Quantitative instruments must be rigorously assessed to ensure their relevance and utility in
identifying the psychological characteristics that most influence performance (Haynes et al.,
2018). Reliable and valid measures enhance assessment accuracy, enabling the identification of
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 7
key psychological predictors and supporting the development of tailored training programs that
meet cadets' unique needs (Brunyé et al., 2024). Such programs not only improve training
effectiveness and readiness but also foster long-term success and retention by aligning
development pathways with individual psychological profiles (Seligman & Fowler, 2011).
This study employed 13 established psychological instruments selected for their proven
relevance in military, sport, and academic contexts. While these measures have demonstrated
validity in various populations, their use within the ROTC population remains limited. Among
these 13 instruments, only a subset has been tested in military populations (e.g., Barbour, 2014;
Gucciardi et al., 2021; Kastle et al., 2021; Rice et al., 2013). Furthermore, only four studies
utilizing these scales have been conducted within the U.S. (Barbour, 2014; Francis, 2021; Kastle
et al., 2021; Seipel, 2020), with a singular study employing one of these tools among non-U.S.
military cadets (Lepinoy et al., 2023).
The limited use of psychological instruments within ROTC populations raises important
concerns about their applicability and accuracy in this unique context. Psychological constructs
such as motivation, resilience, and stress may manifest differently in ROTC cadets compared to
other military populations due to their transitional developmental stage and the interplay of
multifaceted responsibilities (Baldwin & Young, 2020). Unlike active-duty soldiers who operate
within a singular military framework, ROTC cadets navigate a hybrid environment where
civilian, academic, and military roles intersect. This intersection introduces variability in
psychological experiences that existing instruments, often validated in more homogenous
military or academic samples, may not adequately capture (Haynes et al., 2018; He & van de
Vijver, 2012). For example, the construct of motivation in cadets may be shaped by the
competing demands of completing academic degrees while simultaneously adapting to military
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 8
leadership structures (Raabe et al., 2020). Similarly, constructs like burnout—linked to
performance and retention in military settings—may have distinct antecedents and
manifestations in cadets due to their unique stressors (Hosseini et al., 2023). Instruments
designed for active-duty personnel often overlook the dual civilian-military realities of cadets,
risking biased or inaccurate assessments of psychological traits essential for success on the
ACFT and in future military careers. These gaps underscore the need for more tailored
approaches to measuring psychological constructs in this population.
Accurate measurement of psychological constructs in ROTC cadets has direct
implications for readiness, retention, and leadership development. Validating these instruments
ensures they reliably identify key predictors of performance, such as mental toughness,
resilience, and motivation regulation, which may be critical for success on the ACFT and broader
military careers (Gucciardi et al., 2021; Brunyé et al., 2024). Following best practices in
psychometric validation is essential for ensuring these tools are accurate and applicable to
diverse populations (Boateng et al., 2018). Given the Army’s reliance on data-driven
interventions to improve training and reduce attrition (e.g., Royston et al., 2022; Russel et al.,
2023), validating these instruments aligns with organizational goals of readiness and leadership
development. Without such evaluations, the risk of inaccurate assessments undermines efforts to
optimize training, enhance cadet success, and support their progression into effective military
leaders (Baldwin & Young, 2020; Flood & Keegan, 2022).
Therefore, this study aims to investigate the psychological factors that may influence
ROTC cadets' ACFT performance, focusing first on the suitability of existing assessment tools
and then on identifying specific psychological traits linked to performance outcomes.
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 9
[RQ1] What are the psychometric properties of existing instruments measuring relevant
psychological characteristics of ROTC cadets?
[RQ2] What psychological characteristics are associated with ROTC cadets’ performance
on the ACFT?
Method
Participants
The study sample consisted of N = 180 ROTC cadets with an average age of 19.99 (±
2.07) years. There were n = 72 (40.0%) Military Science (MS) I cadets, n = 42 (23.3%) MSII
cadets, n = 36 (20.0%) MSIII cadets, and n = 30 (16.7%) MSIV cadets. Most participants
identified as male (n = 145, 80.56%), with 33 cadets identifying as female (18.33%) and two
(1.1%) preferring not to disclose their gender. Most cadets identified as White/Caucasian (n =
149, 82.78%), followed by smaller groups identifying as Black or African American (n = 8,
4.44%), Asian (n = 4, 2.22%), Hispanic or Latino (n = 6, 3.33%), and multi-racial (n = 11,
6.11%); two participants (0.56%) preferred not to disclose their race. A total of 65 cadets
(36.11%) held an ROTC scholarship, while 115 (63.89%) did not; 72 cadets (40.00%) reported
prior enlistment experience, whereas 108 (60.00%) did not (see Table 1).
Study Design
Following Institutional Review Board approval, ROTC cadets actively enrolled at the
first author’s university were invited to participate in this study. ROTC cadets were recruited
using a convenience sampling approach. Approval was first obtained from the ROTC cadre, after
which we were granted permission to recruit participants in person during military science
courses and pre-ACFT procedures, such as cadre-led cadet weigh-ins. Participants were
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 10
informed that their involvement in the study was entirely voluntary and that all data collected
would remain anonymous and confidential.
The current study employed a cross-sectional design in which data was collected at four
total time points over a two-year academic period: time point one (T1) occurred in Fall of 2022,
time point two (T2) occurred in Spring of 2023, time point three (T3) occurred in Fall of 2023,
and time point four (T4) occurred in Spring of 2024. However, only the initial time point for
each participant was used in the analysis. Of the 427 total cadets who were contacted, 334 cadets
agreed to participate (78.2% response rate). In line with the research questions for the current
research, only the data of participants’ first participation in the study were analyzed for this study
(N = 180).
Measures
The measures employed in the current study comprised 244 items and took
approximately 30 minutes to complete. The following measures were presented in the order
provided below and remained consistent across all data collection time points. Of the 244 survey
items, eight questions collected demographic information on participants, including their age,
gender, race, cadet class, years as a cadet, years in ROTC, ROTC scholarship status, and prior
military enlistment.
Army Combat Fitness Test (ACFT)
ACFT data was collected via official scorecards filled out and submitted by the ROTC
cadre (see Appendix O). The ACFT is comprised of a max deadlift (three repetitions), a standing
power throw, hand-release push-ups, a spring/drag/carry, a plank, and a two-mile run (U.S.
Department of the Army, 2023; see Appendix P). All scoring in the ACFT events is normed by
age group and gender (e.g., 17-21 years old, male, and female) and each event has a total
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 11
possible score of 100 points. The max deadlift (3RMD) requires three successful repetitions of
140 pounds for males and 120 pounds for females regardless of age group. The standing power
throw (STP), in which a cadet tosses a 10-pound medicine ball for maximum distance, requires a
6.0-meter and 3.9-meter score for males and females respectively in the 17–21-year-old age
group. The hand-release push-up (HRP) event requires 10 successful repetitions of the push-up
to pass regardless of age or gender. The sprint/drag/carry (C) event requires cadets to first sprint
50 meters, then drag a 90-pound sled 50 meters, then laterally sprint 50 meters, then carry two
40-pound kettlebells 50 meters, and lastly spring another 50 meters. Passing scores for the
sprint/drag/carry event vary by age and gender, with a 2:28 minute and a 3:15 minute passing
time for male and female cadets aged 17-21. The plank (P), also scored by time and varied by
age and gender, has a passing score of achieving at least 1:30 seconds for male and female cadets
aged 17-21. Lastly, the two-mile run (2MR) requires cadets to complete the event in under 22:00
minutes for males and under 23:22 minutes for females aged 17-21. To officially pass the ACFT,
cadets are required to score a minimum of 60 points on each event and an overall raw score of at
least 360 points.
Academic Motivation Scale – College (AMS-C)
The AMS-C (Vallerand et al., 1992) is a 28-item instrument measuring the degree of
motivational regulation in the context of a university across seven subscales including:
amotivation (four items; e.g., “”I don’t know; I can’t understand what I am doing in school”),
extrinsic motivation – external regulation (four items; e.g., “in order to obtain a more prestigious
job later on”), extrinsic motivation – introjected regulation (four items; e.g., “because I want to
show myself that I can succeed in my studies”), extrinsic motivation – identified regulation (four
items; e.g., “because this will help me make a better choice regarding my career orientation”),
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 12
intrinsic motivation – to experience stimulation (four items; e.g., “for the intense feelings I
experience when I am communicating my own ideas to others”), intrinsic motivation – toward
accomplishment (four items; e.g., “for the pleasure I experience while surpassing myself in my
studies”), and intrinsic motivation – to know (four items; e.g., “because my studies allow me to
continue to learn about many things that interest me”). Each item is rated on a seven-point Likert
scale from 1 (does not correspond at all) to 7 (corresponds exactly) with a mid-point of 4
(corresponds moderately). To score the AMS-C, the responses for each subscale are first
summed for each participant. A Self-Determination Index (SDI) is then calculated to determine
the degree to which participants are self-determined in college, ranging from -18 (low selfdetermination) to +18 (high self-determination). The SDI is calculated by assigning weights to
the motivation regulation subscales, multiplying each subscale’s mean score by its corresponding
weight (Howard et al., 2020). The weighted scores across all subscales are then summed to
compute the total SDI. Higher SDI values indicate greater self-determined motivation, while
lower values suggest controlled or amotivated forms of motivation (Howard et al., 2020).
Achievement Motives Scale – Revised (AMS-R)
The AMS-R (Lang & Fries, 2006) is a 10-item questionnaire that assesses participants’
achievement motives across two subscales: hope for success (five items; e.g., “I like situations in
which I can find out how capable I am”) and fear of failure (five items; e.g., “I feel uneasy to do
something if I am not sure of succeeding”). Participants respond on a four-point Likert scale
from 1 (strongly disagree) to 4 (strongly agree). Scores for each achievement motive are
calculated by averaging items for each subscale.
Basic Needs Satisfaction at College Scale (BNSC-S)
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 13
The BNSC-S (Jenkins-Guarniere et al., 2015) is a 13-item instrument measuring
students’ need satisfaction in a university context across three subscales: autonomy satisfaction
(four items; e.g., “I feel like I can pretty much be myself at school”), competence satisfaction
(five items; e.g., “I do not feel very competent with school”), and relatedness satisfaction (four
items; e.g., “I really like the people I go to school with”). Each item is rated on a five-point
Likert scale from 1 (not at all true) to 5 (very true), with a mid-point of 3 (somewhat true). The
score for each basic psychological need subscale is computed by averaging the respective items.
Basic Psychological Need Satisfaction and Frustration Scale (BPNSFS)
The BPNSFS (Chen et al., 2015) is a 24-item scale measuring participants’ basic
psychological need fulfillment across six subscales: autonomy satisfaction (four items; e.g., “I
feel my choices express who I really am”), autonomy frustration (four items; e.g., “my daily
activities feel like a chain of obligations”), competence satisfaction (four items; e.g., “I feel
capable at what I do”), competence frustration (four items; e.g., “I feel disappointed with many
of my performances”), relatedness satisfaction (four items; e.g., “I feel that the people I care
about also care about me”), and relatedness frustration (four items; e.g., “I feel the relationships I
have are just superficial”). Participants answer each item on a five-point Likert scale from 1 (not
true at all) to 5 (completely true). The scoring of the BPNSFS can be computed in many ways
(see Van der Kaap-Deeder et al., 2020). For instance, need satisfaction (i.e., autonomy,
competence, and relatedness satisfaction) and need frustration may be scored separately by
averaging the respective items to be then contrasted within the sample. To best fit the sample, the
stem of the scale was changed from “we ask about the kind of experiences you actually have in
your life” to “we ask about the kind of experiences you actually have in your life as a cadet.”
Cognitive and Somatic Anxiety Inventory – 2 Revised (CSAI-2R)
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 14
The CSAI-2R (Cox et al., 2003) is a 17-item instrument assessing participants’ state
anxiety and confidence levels regarding a specific task across three subscales: cognitive anxiety
(five items; e.g., “I’m concerned about performing poorly”), somatic anxiety (seven items; e.g.,
“I feel jittery”), and self-confidence (five items; e.g., “I’m confident I can meet the challenge”).
Each item is rated on a four-point Likert scale from 1 (not at all) to 4 (very much so), with midpoints of 2 (somewhat) and 3 (moderately so). To better fit the sampled participants, the stem of
the CSAI-2R included “Army Combat Fitness Test (ACFT)” (e.g., “indicate how you typically
feel right before your Army Combat Fitness Test (ACFT)”). The CSAI-2R is scored by first
averaging each subscale for each participant and then multiplying the averages by 10.
Coping Function Questionnaire (CFQ)
The CFQ (Kowalski & Crocker, 2001) is an 18-item instrument that assesses individuals’
use of different coping functions across three subscales: problem- (six items; e.g., “I work harder
to try to change the situation”), emotion- (seven items; e.g., “I try to find ways to control my
emotions”), and avoidance-focused coping (five items; e.g., “I try to get away from the situation
to reduce the stress”). Participants respond on a five-point Likert scale from 1 (not at all) to 5
(very much). Scores for each coping function are calculated by averaging items for each
subscale.
Locus of Control Scale (LCS)
The LCS (Rotter, 1966) is a 29-item instrument that measures participants’ degree of
external versus internal control of reinforcement by employing dichotomous questions (e.g.,
“many of the unhappy things in people’s lives are partly due to bad luck” versus “people’s
misfortunes result from the mistakes they make”). The survey is scored by summing scores on all
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 15
items, in which a high score corresponds to greater levels of an external locus of control, whereas
a low score corresponds to greater levels of an internal locus of control.
Mental Toughness Inventory (MTI)
The MTI (Gucciardi et al., 2015) is an eight-item instrument assessing a person’s
perceived mental toughness (e.g., “I consistently overcome adversity”). Each item is rated on a
seven-point Likert scale from 1 (false, 100% of the time) to 7 (true, 100% of the time). To better
fit the sampled participants, the stem of the MTI was adapted from “how you think, feel, and
behave as an athlete” to “how you think, feel, and behave as a cadet.” The score for the entire
scale was calculated by averaging responses to all items, in which lower scores represent smaller
indices of mental toughness for cadets.
Military Academic Motivation Scale (MAMS)
The MAMS (Filosa et al., 2021) is a 17-item instrument that measures cadets’ military
academy motivation regulation across five subscales: amotivation (four items; e.g., “I am in
ROTC just because I have to”), external regulation (four items; e.g., “because I want to move up
and make lots of money”), introjected regulation (four items; e.g., “to show that I’m able to do
anything, if I want to), identified regulation (four items; e.g., “to achieve the position I want in
the future”, and intrinsic motivation (four items; e.g., “because in ROTC I can do things that I
like”). Each item is rated on a five-point Likert scale from 1 (not at all) to 5 (very much). To
better fit the sampled participants, the stem of the MAMS was adapted from “why did you
decide to join the Guardia di Finanza?” to “why you are in ROTC.” Additional changes were
made to six items by substituting the words “Guardia di Finanza” with “ROTC”. Scoring for the
MAMS is equivalent to that of the AMS-C, in which an SDI is calculated to determine the
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 16
degree to which participants are self-determined in their military academy setting (Howard et al.,
2020).
Oldenburg Burnout Inventory (OLBI)
The OLBI (Halbesleben & Demerouti, 2005) is a 16-item questionnaire that measures
burnout across two subscales: disengagement (eight items; e.g., “I find my work to be a positive
challenge”) and exhaustion (eight items; e.g., “when I work, I usually feel energized”).
Participants respond to each item on a four-point Likert scale from 1 (strongly agree) to 4
(strongly disagree). The instrument is scored by first summing the two subscales, and then by
summing the two subtotals, in which higher total scores are associated with greater levels of
burnout.
Perceived Stress Scale – 10 (PSS-10)
The PSS-10 (Cohen & Williamson, 1988) is a 10-item survey measuring participants’
frequency of perceived stress (e.g., “in the last month, how often have you felt nervous and
‘stressed’”?). Responses to each item fall on a five-point Likert scale from 0 (never) to 4 (very
often), with a mid-point of 2 (sometimes). The total score is calculated by adding the scores for
all 10 items, in which higher total scores are indicative of higher levels of perceived stress.
Resilience Scale (RS)
The RS (Wagnild & Young, 1993) is a 25-item instrument that measures the resilience of
participants (e.g., “I take things one day at a time”). Items in the RS are rated on a seven-point
Likert scale from 1 (strongly disagree) to 7 (strongly agree). The RS is scored by summing the
scores across all items, in which larger scores are associated with higher levels of resilience.
Task and Ego Orientation in Sport Questionnaire (TEOSQ)
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 17
The TEOSQ (Duda & Nicholls, 1992) is a 21-item instrument that measures the degree of
participants’ beliefs about the causes of success across four subscales: ego orientation (eight
items; e.g., “others can’t do as well as me”), task orientation (eight items; e.g., “I learn
something interesting”), work avoidance (three items; e.g., “I don’t have to try hard”), and
cooperation (two items; e.g., “my friends and I help each other”). Each item is rated on a fivepoint Likert scale from 1 (strongly agree) to 5 (strongly disagree). The score for each subscale is
computed by averaging the respective items.
Data Analysis
The data was analyzed using SPSS (version 26) and R Studio (version 4.4.1). Preliminary
analyses included an evaluation of missing data patterns and outlier detection. Little’s MCAR
test (Little, 1988) was conducted to determine if the missing data were missing completely at
random, justifying listwise deletion for subsequent analyses. Cases with values beyond ± 3
standard deviations from the mean were identified as outliers and removed from the dataset to
mitigate potential distortions in the analysis, following recommended practices to prevent undue
influence by extreme values (Tabachnick & Fidell, 2019).
To examine the psychometric properties of the instruments, internal consistency
reliability was assessed for each scale and subscale using Cronbach’s alpha. Cronbach’s alpha
values provide an estimate of reliability by measuring the extent to which items within each
instrument are consistent with one another, with values closer to 1.0 indicating higher reliability.
According to widely accepted guidelines, alpha values above 0.90 are considered excellent,
values between 0.80 – 0.89 are good, values between 0.70 – 0.79 are acceptable, values between
0.60 – 0.69 are questionable, and values below 0.60 are poor (Taber, 2018).
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 18
Descriptive statistics were computed to summarize the characteristics of the study sample
and the distribution of the variables of interest. Confirmatory Factor Analysis (CFA) was
conducted using R Studio with the Weighted Least Squares Mean and Variance Adjusted
(WLSMV) estimator. This estimator, a type of Diagonally Weighted Least Squares (DWLS)
estimator, is appropriate given the ordinal nature of the data and the sample size (n < 200). The
WLSMV estimator accounts for potential heteroscedasticity and non-normality, providing
greater accuracy in parameter estimates and model fit assessments (DiStefano et al., 2019).
Model fit was evaluated following the recommendations laid out by Hu and Bentler
(1999) using scaled fit indices, including Comparative Fit Index (CFI) values close to 0.95,
Tucker-Lewis Index (TLI) values close to 0.95, Root Mean Square Error of Approximation
(RMSEA) values close to 0.08, and Standardized Root Mean Square Residual (SRMR) values
close to 0.06 (DiStefano et al., 2019). Factor loadings of 0.40 or higher were considered
acceptable, as recommended by Hooper et al. (2008), to ensure items meaningfully represented
the latent constructs.
Pearson correlation coefficients were computed to examine the bivariate relationships
between each psychological characteristic and performance on each component of the ACFT and
ACFT totals. Correlation analyses were used to identify potential associations between
psychological traits and physical performance among ROTC cadets. Subsequent multiple linear
regression modeling was performed to determine the extent to which psychological
characteristics predicted performance on the components of the ACFT. ACFT sub scores served
as the dependent variable, while the non-collinear psychological characteristics measured by the
scales served as independent variables. Regression coefficients and associated p-values were
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 19
examined to assess the strength and significance of the relationships between psychological traits
and ACFT performance.
Results
Psychometric Properties of Psychological Instruments (RQ1)
The internal consistency of each instrument and the respective subscales was evaluated
using Cronbach’s alpha. Following conventional reliability thresholds (i.e., Taber, 2018),
analyses confirmed that most instruments and subscales possessed satisfactory internal
consistency, supporting their use in psychological characteristics among ROTC cadets (see Table
2). Specifically, among the 13 total instruments and 35 scales (48 total), six were found to be
poor or questionable: the BNSC-S competency satisfaction subscale (α = 0.51), the MAMS
identified regulation subscale (α = 0.59), the TEOSQ work avoidance subscale (α = 0.65), the
BNSC-S autonomy satisfaction subscale (α = 0.65), the LCS scale (α = 0.66), and the MAMS
amotivation subscale (α = 0.68).
Model fit indices and factor loadings from the confirmatory factor analyses are presented
in Table 3 for all 13 instruments. Among these, five instruments demonstrated acceptable model
fit and were included for further analysis: Academic Motivation Scale-College (AMS-C; CFI =
0.95, TLI = 0.95, RMSEA = 0.08, SRMR = 0.07, 0.58 ≤ λ ≤ 0.96), Achievement Motives ScaleRevised (AMS-R; CFI = 0.97, TLI = 0.97, RMSEA = 0.08, SRMR = 0.07, 0.45 ≤ λ ≤ 0.90),
Cognitive and Somatic Anxiety Inventory-2 Revised (CSAI-2R; CFI = 1.00, TLI = 0.99,
RMSEA = 0.05, SRMR = 0.05; 0.75 ≤ λ ≤ 0.97), Resilience Scale (RS; CFI = 0.94, TLI = 0.94,
RMSEA = 0.08, SRMR = 0.07, 0.36 ≤ λ ≤ 0.77), and Basic Psychological Need Satisfaction and
Frustration Scale (BPNSFS; CFI = 0.97, TLI = 0.96, RMSEA = 0.07, SRMR = 0.07, 0.58 ≤ λ ≤
0.93).
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 20
The initial CFA of the RS resulted in three items below 0.40 (i.e., 0.01, 0.39, 0.46). The
item with a factor loading of 0.01 was removed to enhance model fit, while the remaining lowloading items were retained to preserve the integrity of the original scales (Hooper et al., 2008).
Psychological Predictors of ACFT Performance Among ROTC Cadets (RQ2)
The descriptive statistics for the psychological instruments administered to the cadets and
correlational analysis among the psychological variables and ACFT events are presented in
Tables 17 and 18.
For each ACFT event, regression analyses were conducted to examine the predictive role
of psychological factors. Significant models are presented first, followed by models without
significant main effects.
The regression model for the two-mile run (see Table 26) was significant, explaining
33% of the variance (R
2 = 0.33, Ajd. R
2 = 0.17, F(19, 148) = 1.91, p < .05). Altogether, five
psychological constructs positively predicted performance on the 2MR: Amotivation (β = 0.41, p
< .05), Identified Regulation (β = 0.42, p < .05), Intrinsic Regulation – Toward Accomplishment
(β = 0.35, p < .05), Cognitive Anxiety (β = 0.31, p < .05), and Self-Confidence (β = 0.33, p <
.05). Both External Regulation (β = -0.53, p < .05) and Intrinsic Regulation – To Know (β = -
0.33, p < .01) were significant negative predictors of cadet performance on the 2MR.
The regression model for the deadlift (see Table 21) was significant, explaining 37% of
the variance (R
2 = .37, Ajd. R
2 = .22, F(19, 148) = 2.13, p < .01). Within this model, Intrinsic
Regulation to Experience Stimulation was a significant positive predictor of performance (β =
0.33, p < .05).
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 21
The regression model for the hand-release push-ups (see Table 23) was significant,
explaining 32% of the variance (R
2 = 0.32, Ajd. R
2 = 0.16, F(19, 148) = 2.12, p < .05).
Resilience was a significant positive predictor of performance (β = 0.21, p < .05).
The regression model for the plank (see Table 25) was significant, explaining 33% of the
variance (R
2 = 0.33, Ajd. R
2 = 0.18, F(19, 148) = 1.91, p < .05). Fear of Failure (β = -0.36, p <
.05) negatively predicted performance, whereas Self-Confidence (β = 0.24, p < .05) positively
predicted plank scores.
Both the standing power throw (R
2 = 0.25, Ajd. R
2 = 0.07, F(19, 148) = 1.91, p > .05)
and the sprint-drag-carry (R
2 = 0.28, Ajd. R
2 = 0.11, F(19, 148) = 1.68, p > .05) models did not
reach significance. (see Tables 22 and 24).
Discussion
This study evaluated the psychometric properties of multiple psychological instruments
in an ROTC cadet population (RQ1) and explored the relationships between these psychological
constructs and ACFT performance (RQ2). The first section of this discussion provides a
discussion of the key findings relative to RQ1, while the second section explores and integrates
the results of RQ2 analyses. This section will conclude with a discussion of limitations and
future directions for research.
Psychometric Properties of Psychological Instruments (RQ1)
Included Instruments
The psychometric evaluation of 13 instruments within the ROTC cadet population
provided insights into their strengths and limitations in predicting ACFT outcomes. Five
instruments – AMS-C, AMS-R, BPNSFS, CSAI-2R, and RS – demonstrated acceptable internal
consistency, model fit, and factor loadings, suggesting these measures effectively captured their
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 22
respective constructs in this sample. These results provide initial evidence that these scales can
yield valid and reliable measurements, which may support future efforts to identify cadet
characteristics aligned with ROTC training methods.
To the author’s knowledge, these five instruments’ internal consistency results have been
provided in a few prior studies with military samples, yet no factor loadings or fit indices were
described. Across the six studies which reported Cronbach’s alpha of instruments among military
participants, the current internal consistency of the AMS-C (Buekers, 2023; 2024; Seipel, 2020),
the AMS-R (Cheng & Magraw-Mickelson, 2023), the BPNSFS (Lepinoy et al., 2023), the CSAI2R (Sánchez-Molina et al., 2019) and the RS (Francis, 2021) are in alignment. Notably, six
studies utilizing the AMS-C, AMS-R, BPNSFS, CSAI-2R, or RS did not report psychometric
properties (Barbour, 2014; Clemente-Suárez et al., 2016; Fuentes-García et al., 2021; Kastle et
al., 2021; Sánchez-Molina et al., 2019; Wesemman et al., 2018). The scarcity of reported
psychometric values presents challenges in ascertaining consistency with other tested military
samples.
Thus, it is necessary to juxtapose the current results across literature that sampled nonmilitary personnel. The AMS-C retained its seven-factor structure with acceptable loadings and
fit, consistent with previous validations across diverse cultures, languages, and educational
contexts (e.g., Akoto, 2014). However, alternate structures, such as the three-factor model
reported by Colebrusco de Souza et al. (2021), suggest cultural or contextual influences and
preferences for streamlined models, warranting careful evaluation in ROTC settings.
Similarly, the AMS-R demonstrated factor loadings and fit indices consistent with its
initial validation (Lang & Fries, 2006). However, the lack of subsequent studies validating the
AMS-R in diverse contexts highlights a significant research gap, underscoring the current
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 23
study’s contribution to its applicability in military populations. Future research should replicate
these findings with larger, more varied samples.
The BPNSFS supported a six-factor model, in line with previous studies validating the
distinction between need satisfaction and frustration (e.g., Chen et al., 2015). The corroborative
support of the six-factor model, as opposed to the two-factor model (i.e., need satisfaction and
need frustration), is further evidenced by the current results and may confirm the theoretical
nature of basic psychological need satisfaction and frustration as distinct constructs (Chen et al.,
2015). In the context of ROTC cadets, the ability to differentiate between need satisfaction and
frustration may provide critical insights into how these constructs influence other factors (e.g.,
motivation, and performance). Future research is needed to validate the six-factor model in other
ROTC cadet and military populations.
The CSAI-2R demonstrated robust psychometric properties, consistent with prior
research validating its three-factor model across languages and populations, including athletes
and children (e.g., Martinent et al., 2010; Stadulis et al., 2002). The consistency of these results
highlights the robustness of the CSAI-2R in measuring performance-related psychological
constructs across diverse populations. In the current study, the CSAI-2R demonstrated
psychometric properties consistent with prior findings, suggesting its applicability to high-stakes,
structured environments like ROTC training.
Finally, the RS displayed adequate reliability and validity across various populations and
languages (e.g., Seong et al., 2024; Serrão et al., 2021). However, ongoing disputes about its
dimensionality, with some studies supporting a unidimensional model (Serrão et al., 2021) and
others a five-factor structure (Konaszewski et al., 2021), underscore the need for further
investigation. Notably, this study identified a problematic item consistent with prior findings,
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 24
suggesting potential context-specific challenges. These somewhat contradictory results across the
literature may be a consequence of cultural context (He & van de Vijver, 2012), methodological
approach (Podsakoff et al., 2012), or item interpretation (DeVellis, 2016). For instance, it is
possible that differences in statistical methods, such as exploratory factor analysis, may yield
variations in the scale’s dimensionality. Yet, exploratory factor analysis, nor exploratory
structural equation modeling, was employed in the current study, so further interpretation of the
dimensionality variability of the RS in the current population is challenging.
This study appears to be the first to assess ROTC cadets using these five instruments,
emphasizing the need for additional psychometric reporting to reinforce evidence of reliability
and validity within diverse military samples. Given the limited sample size and the specificity of
the population examined here, future research should aim to replicate these findings across larger
and more diverse military samples. Expanding the sample size and scope would allow for more
robust testing of model fit and factorial invariance, helping to establish greater generalizability
(Kline, 2016). Additionally, longitudinal studies could explore the stability of these instruments'
psychometric properties over time and in varied military training contexts.
Excluded Instruments
The remaining eight scales (i.e., BNSC-S, CFQ, LCS, MAMS, MTI, OLBI, PSS,
TEOSQ) did not demonstrate adequate psychometric properties in this study, suggesting
potential misalignment between these instruments and the current sample population. Particular
items across four of the excluded scales presented poor factor loadings. In the context of the
current study, low item factor loadings may be indicative of a myriad of reasons, such as item
irrelevance for the population (He & van de Vijver, 2012), poor item-construct alignment
(Putnick & Borstein, 2016), complex or ambiguous wording of items (Clark & Watson, 2019),
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 25
lack of variability in responses (DeVellis, 2016), multidimensionality of constructs (Reise et al.,
2007), small sample size, and possible random measurement error (Podsakoff et al., 2012).
Specifically, two items from the BNSC-S (e.g., “I have been able to learn interesting new
skills in college”), two items from the OLBI (e.g., “There are days when I feel tired before I
arrive at work”), 14 items from the LCS (e.g., “One of the major reasons why we have wars is
because people don’t take enough interest in politics”), one item from the TEOSQ (e.g., “I can
goof off”), and one item from the MAMS (e.g., “Honestly, I joined ROTC because I felt
compelled”) had factor loadings below 0.40. These poor item factor loadings might be
suggestive of potential issues with content validity (Boateng et al., 2018), in which items may
have weak associations with the underlying factor and, possibly, poor representation of the latent
constructs (Hooper et al., 2008). Given the current study sample, (i.e., public university, a high
proportion of 1st year cadets) for whom participation may have occurred in their first semester in
university, participants may not have had enough exposure to university or ROTC experiences,
possibly reducing the relevance of items and potentially impacting factor loading (He & van de
Vijver, 2012). The fact that other instruments (e.g., AMS-C, AMS-R, BPNSFS, CSAI-2R, RS)
maintained their psychometric integrity despite the sample characteristics may be due to the
nature of the constructs they measure. Constructs like academic motivation may be more stable
and less dependent on specific experiences, resulting in robust psychometric properties even for
1st-year cadets. Additionally, instruments like the AMS-C and RS have undergone extensive
validation, enhancing their reliability across diverse contexts, which may account for their
stronger performance in the current study.
As seen with the BNSC-S in the current study, it is probable that the items with low
factor loadings did not contribute strongly to the factor they are meant to measure (e.g.,
PSYCHOMETRICS & PREDICTORS OF ROTC PERFORMANCE 26
Competence Satisfaction), or destabilized the factor structure, resulting in fit indices that indicate
poorer fit (Heene et al., 2011). Item irrelevance for the population may have also been evidenced
within the Work Avoidance subscale of the TEOSQ, originally developed in sport (Duda &
Nicholls, 1992), which may be interpreted differently in academic or ROTC settings. This impact
on the interpretation of items may, through item irrelevance, affect factor structure. In contrast,
the relatively greater psychometric performance of instruments like the AMS-C, AMS-R,
BPNSFS, CSAI-2R, and RS may reflect their development in a context that aligns with the
ROTC environment. For example, before efforts in validating the BPNSFS across a wide variety
of contexts, the instrument was originally tested primarily among undergraduate students (Chen
et al., 2015; Cordeiro et al., 2016), likely allowing for a degree of universality among ROTC
cadets who must