Cognitive Behaviour Therapy

ISSN: 1650-6073 (Print) 1651-2316 (Online) Journal homepage: http://www.tandfonline.com/loi/sbeh20

A longitudinal assessment of the road to mental
readiness training among municipal police
R. Nicholas Carleton, Stephanie Korol, Julia E. Mason, Kadie Hozempa,
Gregory S. Anderson, Nicholas A. Jones, Keith S. Dobson, Andrew Szeto &
Suzanne Bailey
To cite this article: R. Nicholas Carleton, Stephanie Korol, Julia E. Mason, Kadie Hozempa,
Gregory S. Anderson, Nicholas A. Jones, Keith S. Dobson, Andrew Szeto & Suzanne Bailey
(2018): A longitudinal assessment of the road to mental readiness training among municipal police,
Cognitive Behaviour Therapy, DOI: 10.1080/16506073.2018.1475504
To link to this article: https://doi.org/10.1080/16506073.2018.1475504

© 2018 The Author(s). Published by Informa
UK Limited, trading as Taylor & Francis
Group
Published online: 18 Jun 2018.

Submit your article to this journal

Article views: 93

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=sbeh20

Cognitive Behaviour Therapy, 2018
https://doi.org/10.1080/16506073.2018.1475504

OPEN ACCESS

A longitudinal assessment of the road to mental readiness
training among municipal police
R. Nicholas Carletona , Stephanie Korola, Julia E. Masona , Kadie Hozempaa
Gregory S. Andersonb, Nicholas A. Jonesc, Keith S. Dobsond, Andrew Szetod and
Suzanne Baileye

,

a

Anxiety and Illness Behaviour Laboratory, Department of Psychology, University of Regina, Regina, Canada;
Justice Institute of British Columbia, New Westminster, Canada; cDepartment of Justice Studies, University of
Regina, Regina, Canada; dDepartment of Psychology, University of Calgary, Calgary, Canada; eSSO Social Work
& MH Trg, CF Health Services Group HQ, Canadian Armed Forces, Ottawa, Canada
b

ABSTRACT

Police agencies increasingly implement training programs to protect
mental health. The Road to Mental Readiness (R2MR) program
was designed by the Canadian military to increase mental health
resilience. A version of R2MR was adapted for municipal police
by the Mental Health Commission of Canada (MHCC). The current
research was designed to assess the R2MR program, as adapted and
delivered by the MHCC, in a municipal police sample. Participants
were 147 Canadian police agency employees (57% women) who
received a single R2MR training session. Participants completed preand post-training self-report questionnaires, and follow-ups at 6 and
12 months. The questionnaires assessed mental health symptoms,
work engagement, resiliency, mental health knowledge, and stigma.
Multilevel modeling analyses assessed for within-participant changes
over time. The results were consistent with other single session
interventions; specifically, there were no significant changes in mental
health symptoms, resilience, or work engagement (p > .05). There were
small, but significant (p < .05), reductions in stigma at post-training
that may facilitate help-seeking among police; relatedly, in openended response fields, participants commonly described the training
as helpful for changing attitudes and improving communication. More
engagement with the material may produce larger, sustained gains,
but more published research is critically needed.

ARTICLE HISTORY

Received 4 February 2018
Accepted 24 April 2018
KEYWORDS

Resiliency; training; road to
mental readiness (R2MR);
stigma

Introduction
Police officers are regularly exposed to many diverse stressors within their organization
(e.g. interference with family life, excessive workload, uncivil interactions with coworkers;
Adams & Buck, 2010; Collins & Gibbs, 2003; Davey, Obst, & Sheehan, 2001; Karaffa & Koch,

CONTACT R. Nicholas Carleton

Nick.Carleton@uregina.ca

© 2018 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License
(http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium,
provided the original work is properly cited, and is not altered, transformed, or built upon in any way.

2

 R. N. CARLETON ET AL.

2016; Morash, Kwak, & Haarr, 2006; Pasillas, Follette, & Perumean-Chaney, 2006) and in
relation to their job duties (e.g. interactions with hostile suspects, offenders, and emotionally distraught victims; Anderson, Litzenberger, & Plecas, 2002; Garcia, Nesbary, & Gu,
2004). Among police officers, social stressors produced from civilian, suspect, co-worker,
and supervisor interactions, have also been significantly related to reduced productivity,
increased aggressiveness (Karaffa & Koch, 2016), turnover intention, psychological distress,
and emotional exhaustion (Adams & Buck, 2010). The organizational and operational stressors that police officers regularly face can be associated with negative mental health outcomes
(van der Velden, Kleber, Grievink, & Yzermans, 2010) including depression (Wang et al.,
2010), maladaptive coping strategies (i.e. anger repression; Hakan Can & Hendy, 2014),
and posttraumatic stress disorder (PTSD) (Maguen et al., 2009), as well as risk for physical
health problems, burnout, smoking, sleep disorders, alcohol abuse, and suicide ideation
(Anshel, 2000; Brown & Campbell, 1994; Gershon, Barocas, Canton, Li, & Vlahov, 2009;
Karaffa & Koch, 2016; Waters & Ussery, 2007). Karaffa and Koch (2016, p. 760) reported
that “the accumulation of stress can overwhelm even the most resilient officers”.
In a large survey of public safety personnel (PSP; e.g. correctional workers and officers,
call center operators and dispatchers, firefighters, paramedics, police officers), Carleton et
al. (2018) found substantial proportions of PSP self-report symptoms consistent with one
or more mental health concerns (Carleton et al., 2018). Many municipal/provincial police
(36.7%) and Royal Canadian Mounted Police (RCMP; 50.2%) screened positive for at least
one mental disorder (e.g. PTSD, major depressive disorder, generalized anxiety disorder,
social anxiety disorder, panic disorder, alcohol abuse). There is also evidence of high levels
of chronic pain among municipal/provincial police (35.9%) and RCMP (43.4%) (Carleton
et al., 2017), as well as high levels of lifetime suicidal ideation (20.5; 25.7%), planning (8.9;
11.2%), and attempts (2.1, 2.4%) (Carleton et al., in press). Accordingly, there has been a
call for developing a national action plan to address the mental health challenges faced by
Canadian PSP (Oliphant, 2016).
Despite the mental health risks faced by police, evidence suggests cultural stigma and
fear of reprisal appear to be significant barriers to seeking treatment (Wester, Arndt, Sedivy,
& Arndt, 2010). In particular, Karaffa and Koch (2016, p. 761) report police officers’ decision-making regarding seeking mental health assistance can be influenced by a desire to
avoid what they believe are other officers’ perceptions; specifically, that those who seek
help may be unable to perform their duties or “be relied upon for backup”, as well as that
other officers are less likely than they are to seek help. The potential disconnect between
an officer’s personal view of seeking mental health assistance and what they perceive other
officers’ perceptions to be—pluralistic ignorance—can result in additional stress when the
perception potentiates the “out group” in relation to the police subculture (Rose & Unnithan,
2015). Pluralistic ignorance, if left unchecked, can perpetuate officers’ reticence to seek help
(Karaffa & Koch, 2016); however, Millar (2002, p. 163) reported that, contrary to an officer’s
own expectations, those who sought mental health support and shared their experience
received “overwhelming support” from colleagues (Millar, 2002).
In general, the contemporary work environment involves increasing workloads as the
economic pressure to get more from less rises (Houdmont, Randall, Kerr, & Addley, 2013).
As workloads have been progressively getting heavier, workplace stress has increased by
40% and absenteeism has increased by 25% (Houdmont et al., 2013). Increased workload
and workplace pressures have invigorated dialog around the use of workplace training,

COGNITIVE BEHAVIOUR THERAPY 

3

particularly for interventions supporting the development of personal and organizational
resilience (Robertson & Cooper, 2013). Resilience training programs are designed to sustain and protect employee well-being and performance in the face of adversity (Anderson,
Vaughan, & Mills, 2017; Robertson, Cooper, Sarkar, & Curran, 2015). There have been two
systematic reviews (Leppin et al., 2014; Robertson et al., 2015) evidencing the programs as
having small to moderate effect sizes for improving resilience, mental health, and subjective
well-being of participants (e.g. reductions in stress, symptoms of depression, negative affect).
Embedding resilience training that includes evidence-based psychoeducation on stress
and trauma directly into police recruitment and training programs may benefit officers by
reducing stigma and minimizing barriers to seeking treatment (Papazoglou & Andersen,
2014). The Road to Mental Readiness (R2MR) was originally developed by the Canadian
Department of National Defense to “improve short term performance and long term mental health outcomes” through building a “solid foundation in the concept of resilience”
(National Defence & the Canadian Armed Forces, 2017). The R2MR program provides
evidence-based psychoeducation on mental health and stress (i.e. the contemporary Mental
Health Continuum Model where mental health spans Healthy, Reacting, Injured, Ill), as well
as providing a series of evidence-based cognitive behavioral therapy style skills designed to
help participants to manage stress; for example, goal setting, mental rehearsal/visualization,
adapted cognitive monitoring (i.e. awareness of self-talk), and arousal management through
adapted breathing (i.e. tactical breathing). The Canadian Armed Forces report evidence
that the R2MR program is reducing stigma and removing barriers to care seeking among
their members (National Defence & the Canadian Armed Forces, 2017). The Mental Health
Commission of Canada (MHCC) adapted the R2MR material, in collaboration with police
and the Canadian Department of National Defense (DND), for extended use in other
environments, particularly among police, but retaining the same goals of reducing stigma,
reducing barriers to care seeking, improving short-term performance, and supporting longterm mental health (Mental Health Commission of Canada, 2017; Stuart et al., 2014; Szeto
& Adair, 2016); specifically, the R2MR content was modified from the original DND content
by authors Dobson and Szeto, including using videos and photos relevant to police rather
than military, and continually updated thereafter based on feedback from the Calgary police.
Despite the evidence-based foundations for the psychoeducation and skills provided by the
R2MR training programs offered by the Canadian Armed Forces and the Mental Health
Commission of Canada, there is a paucity of research assessing the impact of the program
on PSP. The research results are intended to inform and support the recommended national
PSP mental health action plan (Oliphant, 2016)., including police officers.
The present study was designed to assess the impact of the MHCC R2MR training on several factors in a sample of Canadian municipal police officers. Specifically, the research was
designed to assess for changes in self-reported mental health symptoms, resilience, mental
health awareness, work engagement, and mental health stigma attitudes over a one-year time
frame. We expected that, consistent with prior reports (Leppin et al., 2014; Robertson et al.,
2015), there would be small to moderate improvements in mental health stigma attitudes
immediately after R2MR training and that the improvements would be sustained at 6- and
12-month follow-ups. In addition, we expected small to moderate improvements across all
factors (mental health symptoms, mental health knowledge, resilience, work engagement) at
6- and 12-month follow-ups after R2MR training. The research results are intended to inform
and support the recommended national PSP mental health action plan (Oliphant, 2016).

4

 R. N. CARLETON ET AL.

Method
Procedure
The study was approved by the first author’s University Institutional Research Ethics Board
(File # REB 2015–131). Approximately 570 Canadian police agency employees in a western Canadian city participated in the MHCC R2MR program as part of their training.
All were invited by an administrative member of the police agency to participate in the
current research program to evaluate the impact of the training. An electronic survey was
administered to participants before participating in the R2MR training, immediately after
training, at 6 months post-training, and at 12 months post-training (details below). The
survey was anonymous, with participants self-selecting a unique nondescript identifier that
would allow their responses to be linked over time.
Participants
In total, 147 participants completed the survey prior to receiving the R2MR training
(Mage = 41.52; SD = 8.51). There was a fairly even distribution of females (56.5%) and males
(43.5%) who completed the pre-R2MR survey. Most participants identified as Caucasian
(91.8%), married/cohabitating (74.8%), and heterosexual (96.6%). Most participants identified their rank within the Regina Police Service (RPS) as Constable (46.3%), followed
by Sergeant (25.6%), Corporal (12.2%), Staff Sergeant (8.5%), and Inspector (3.7%). The
remaining participants identified their rank as Deputy Chief, Superintendent, and Special
Constable (3.6%). Most participants indicated that the highest level of education that they
had received was a four-year college/university program (43.8%), followed by a two-year
college program (23.3%), high school/high school equivalent (20.5%), graduate/professional
degree (10.3%), with the remainder not completing high school, or selecting “Rather not
say” (2.1%).
Measures, standardized
Alcohol use disorders identification test (AUDIT; Gache et al., 2005; Saunders, Aasland,
Babor, Delafuente, & Grant, 1993)
The AUDIT is a 10-item self-report measure designed to assess alcohol consumption and
consequences. Items such as “How often do you have six or more drinks on one occasion”
are rated on a five-point Likert-type scale ranging from 1 (Never) to 5 (Four or more times
a week). The AUDIT has received substantial validity and reliability support as a screening
tool for risky alcohol use behaviors (Bohn, Babor, & Kranzler, 1995; Piccinelli et al., 1997;
Volk, Steinbauer, Cantor, & Holzer, 1997).
Brief resilience scale (BRS; Smith et al., 2008)
The BRS is a six-item self-report questionnaire designed to assess ability to adapt and
recover from stressful experiences. Items such as “I tend to bounce back quickly after hard
times” are rated on a five-point Likert-type scale ranging from 1 (Strongly disagree) to 5
(Strongly agree). Half of the items are reverse-coded. The BRS has high internal consistency
(α ≥ .80) and good test–retest reliability (r ≥ .62; Leontjevas, de Beek, Lataster, & Jacobs,
2014; Rodríguez-Rey, Alonso-Tapia, & Hernansaiz-Garrido, 2016; Smith et al., 2008).

COGNITIVE BEHAVIOUR THERAPY 

5

Depression anxiety and stress scale 21-item (DASS-21; Antony, Bieling, Cox, Enns, &
Swinson, 1998)
The DASS-21 is a self-report measure assessing three sets of negative emotional states (i.e.
depression, anxiety, stress), established from the original 42-item questionnaire (Lovibond
& Lovibond, 1995). The Depression scale assesses lack of interest, hopelessness, and low
self-esteem. The Anxiety scale assesses physiological arousal and subjective anxiety. The
Stress scale assesses non-specific arousal, such as, agitation, muscle tension, and negative
affect. Each scale consists of seven items, assessing levels over the past week with a four-point
scale, ranging from 0 (did not apply to me at all) to 3 (applied to me very much, or most of
the time). Overall, the DASS-21 has demonstrated good-to-excellent internal consistency
and convergent validity, acceptable discriminative validity, and a consistent three-factor
structure (e.g. Antony et al., 1998; Clara, Cox, Enns, 2001; Gloster et al., 2008).
Mental health knowledge schedule (MAKS; Evans-Lacko et al., 2010)
The MAKS is a 12-item self-report questionnaire designed to measure mental health literacy
and stigma. Items such as “Most people with mental health problems want to have paid
employment” are rated on a five-point Likert scale ranging from 1 (Strongly disagree) to
5 (Strongly agree). The MAKS is a relatively new measure, but the available psychometric
data support the internal consistency and test–retest reliability of the measure (Evans-Lacko
et al., 2010); in addition, the measure appears sensitive to changes based on interventions
(Hansson & Markstrom, 2014).
Open minds survey of workplace attitudes (OMSWA; Szeto, Luong, & Dobson, 2013)
The OMSWA is a 23-item self-report questionnaire designed to measure mental health
stigma and workplace attitudes. Items such as “I would be upset if a co-worker with a mental
illness always sat next to me at work” are rated on a five-point Likert scale ranging from
1 (Strongly disagree) to 5 (Strongly agree). A single item, “If I knew a co-worker who has
a mental illness, I would not date them”, was omitted per recommendations from Szeto to
accommodate the research sample; as such, only 22 items were used in the current study.
The OMSWA is a relatively new measure, but the available psychometric data support the
internal consistency of the measure (Szeto et al., 2013). The measure is currently in use as
a standard metric by the MHCC.
Posttraumatic stress disorder checklist for DSM-5 (PCL-5; Weathers et al., 2013)
The PCL-5 is a 20-item self-report questionnaire that measures the 20 corresponding DSM-5
symptoms of PTSD. Symptoms experienced in the past month, such as “Feeling very upset
when something reminded you of the stressful experience?” are rated on a 1 (not at all)
to 5 (extremely) Likert-type scale. The PCL-5 has excellent psychometric properties (e.g.
Blevins, Weathers, Davis, Witte, & Domino, 2015; Bovin et al., 2016).
Utrecht work engagement scale-9 items (UWES-9; Schaufeli, Bakker, & Salanova,
2006)
The UWES-9 is a nine-item questionnaire assessing work-related state of fulfillment. Questions
such as “I am proud of the work that I do” are rated on a six-point scale from 0 (never) to 6
(always). The UWES-9 has shown strong internal consistency (αs = .75 to .85) and a more
temporally stable factor structure than longer versions of the UWES (Seppala et al., 2009).

6

 R. N. CARLETON ET AL.

Participants were asked to complete all standardized measures at the baseline (i.e. preR2MR training), 6-month, and 12-month follow-up time points. Participants were asked
to also complete the OMSWA, at post-training (i.e. immediately post-R2MR training) as
well as at the other time points.
Measures, study-specific
R2MR-Specific questionnaire (R2MR-Specific; Tables 2 and 3)
The R2MR-Specific is an 11-item self-report questionnaire designed for the current study
to measure participants’ perceived impact of the R2MR training. The scale contains three
items such as “How helpful did you find the R2MR training?” rated on a five-point scale
ranging from 1 (not at all) to 5 (extremely). Items such as “Have you noticed changes in
your leadership with respect to mental health?” were rated on a five-point scale ranging
from 1 (none at all) to 5 (a great deal). The scale also contained three open-ended questions;
specifically, “What did you find most helpful?”, “What skills will be the most challenging to
implement and why?”, and “What skills will be the easiest to implement and why?”
R2MR-Skills utilization questionnaire (R2MR-Skills; Tables 4 and 5)
The R2MR Skills is a 24-item self-report questionnaire designed for the current study to
measure participants’ utilization of skills taught during R2MR training. The scale contains
nine items prompted by the question, “How often do you use the following skills or resources
from R2MR?” Following the prompt, items such as “Goal setting” are rated on a seven-point
scale ranging from 1 (never) to 7 (always). Participants are asked to use the same seven-point
scale to respond to the question “How often have you assessed where you are on the Mental
Health Continuum Model?”. The scale also contains the same nine items rated on a five-point
scale ranging from 1 (not at all) to 5 (extremely) prompted by the question “How helpful
do you find each of the following skills or resources taught by R2MR?”. A five-point scale
ranging from 1 (strongly disagree) to 5 (strongly agree) was used to rate the five remaining
scale items (e.g. I understand how mental health problems present in the workplace).
Analyses
Data preparation
Descriptive statistics were calculated for each measure. Total scores for each dependent
variable at each time point were calculated. The data was then restructured and a separate
row for each time point per participant was obtained to prepare the data for the primary
analyses described below.
Primary analyses
There were two-level multilevel modeling analyses conducted for each repeated measure
analysis in the current study. Time was coded into three time points including, (1) baseline,
(2) 6-month follow-up, (3) 12-month follow-up, for all dependent variables, except the
OMSWA. Time was coded into four time points for the OMSWA including, (1) baseline, (2)
post-training, (3) 6-month follow-up, (4) 12-month follow-up. Each model included one of
the dependent variables (i.e. AUDIT, BRS, DASS-21 depression, DASS-21 anxiety, DASS21 stress, MAKS, PCL-5, OMSWA, UWES-9) at each time point (level one) nested within

COGNITIVE BEHAVIOUR THERAPY 

7

participants (level two). Baseline differences in scores among individuals were accounted
for by including both a fixed and random intercept in the model. Each final model included
the fixed effect of time as the primary predictor variable at all time points. All models were
computed using the maximum likelihood estimation. All hypothesis testing was conducted
using one-tailed tests at a α level of .05. Cohen’s d effect sizes were computed for all models
by standardizing each outcome measure and rerunning the resulting Z-scores in each model.
All models were bootstrapped using 1000 samples to generate robust probability values and
corresponding confidence intervals.

Results
Descriptive statistics
Descriptive statistics of the dependent variables measured at baseline and at each follow-up
(i.e. 6-month and 12-month) can be found in Table 1. Immediately following the R2MR
training and at the 6-month and 12-month follow-ups, participants also completed the
R2MR-Specific and R2MR-Skills Utilization questionnaires, regarding their views on the
R2MR training, its perceived helpfulness and utility, and its effect on their views of mental
health in the workplace (see Tables 2–6).
Multilevel models
Mental health symptoms
The results of each two-level multilevel model for each mental health symptom measures (i.e.
DASS-21 depression, DASS-21 anxiety, DASS-21 stress, PCL-5, AUDIT), across each time
point (i.e. baseline, 6-month, and 12-month follow-up), are presented in Table 7. Overall,
there were no significant changes in symptoms of depression, anxiety, stress, posttraumatic
stress, and alcohol use, at any follow-up time point, following the R2MR training intervention.
Specifically, compared to both the 6-month follow-up, d = 0.003 (95% CI [−0.41, 0.81]), and
12-month follow-up depression scores, d = 0.038 (95% CI [−0.48, 0.52]), baseline depression
scores were associated with non-significant increases in depression scores. Similarly, when
compared to both the 6-month follow-up, d = 0.129 (95% CI [−0.34, 0.89]), and 12-month
follow-up scores, d = 0.067 (95% CI [−0.35, 0.61]), baseline posttraumatic stress scores were
associated with non-significant increases in posttraumatic stress scores. Additionally, when
baseline anxiety scores were compared to anxiety scores at 6-month, d = 0.023 (95% CI
[−0.50, 0.74]), and 12-month d = −0.20 (95% CI [−0.53, 0.13]), baseline anxiety scores were
associated with non-significant increases in anxiety scores. Furthermore, baseline stress scores
were associated with nonsignificant decreases in stress scores when compared to both the
6-month, d = −0.056 (95% CI [−0.34, 0.23]), and 12-month, d = −0.123 (95% CI [−0.49, 0.36])
follow-up scores. Finally, when compared to baseline alcohol use scores, both the 6-month
follow-up, d = −0.048 (95% CI [−0.25, 0.04]), and 12-month follow-up scores, d = −0.067
(95% CI [−0.42, 0.21]), were associated with non-significant decreases in alcohol use scores.
Mental health awareness, knowledge, and protective factors
The results of each multilevel model evaluating changes in mental health knowledge or
resilience scores (i.e. MAKS, BRS), across each time point (i.e. baseline, 6-month, and

8

 R. N. CARLETON ET AL.

Table 1. Descriptive statistics for dependent measures from baseline to 12-month follow-up.
Time point
Baseline
DASS-21 depression
DASS-21 anxiety
DASS-21 stress
 PCL-5 total
 AUDIT
 UWES-9
BRS
MAKS
 OMSWA
Post-training
 OMSWA
6-month follow-up
DASS-21 depression
DASS-21 anxiety
DASS-21 stress
 PCL-5
 AUDIT
 UWES-9
BRS
MAKS
 OMSWA
12-month follow-up
DASS-21 stress
DASS21 depression
DASS21 anxiety
 PCL-5
 AUDIT
 UWES-9
BRS
MAKS
 OMSWA

n

M

SD

Skewness

Kurtosis

133
133
133
125
110
112
112
112
113

4.00
2.71
5.93
14.78
4.02
32.58
2.95
46.46
2.00

4.44
3.02
4.50
14.33
2.81
8.10
0.29
3.83
0.49

1.37
1.83
1.13
1.21
1.32
−0.57
−0.12
0.22
0.27

1.64
4.88
1.63
1.09
2.54
1.62
1.15
0.83
−0.40

113

1.89

0.44

0.14

−0.29

52
52
52
45
41
44
44
44
45

3.64
2.52
5.15
14.29
4.12
33.09
2.93
46.27
1.98

4.03
3.22
4.15
13.88
2.99
7.18
0.35
3.99
0.54

1.39
2.26
1.46
1.57
2.22
0.32
0.08
−1.06
0.16

1.17
5.96
3.34
2.61
7.51
−0.57
−0.59
4.22
−0.53

56
56
56
44
37
42
42
59
57

5.38
4.21
2.13
14.68
4.24
31.88
2.98
46.53
1.93

4.17
5.32
3.37
15.49
2.96
9.38
0.26
4.13
0.48

0.84
1.75
2.57
1.79
1.35
−0.14
0.38
−0.50
0.56

1.29
2.70
7.53
3.68
3.28
−0.04
0.31
−0.02
0.55

Note: DASS-21 = Depression Anxiety Stress Scales 21-item; PCL-5 = Posttraumatic Stress Disorder Checklist-5; PGI = Posttraumatic Growth Inventory; AUDIT = Alcohol Use Disorders Identification Test; UWES-9 = Utrecht Work Engagement
Scale 9-item; BRS = Brief Resilience Scale; MAKS = Mental Health Knowledge Schedule; OMSWA = Open Minds Survey of
Workplace Attitudes.

12-month), are presented in Table 7. Overall, following R2MR training there were no significant changes in mental health knowledge and resilience scores. Mental health knowledge
scores were associated with non-significant changes at both the 6-month, d = 0.042 (95%
CI [−0.46, 0.74]), and 12-month follow-ups, d = −0.015 (95% CI [−0.36, 0.31]). Similarly,
when compared to baseline resilience scores, both the 6-month follow-up, d = −0.061 (95%
CI [−0.40, 0.28]), and 12-month follow-up scores, d = 0.106 (95% CI [−0.23, 0.45]), were
associated with non-significant changes in resilience scores.
Work engagement and stigma attitudes
The results of each multilevel model of work engagement or workplace stigma attitudes
(i.e. UWES-9, OMSWA) across each time point (i.e. baseline, post-training, 6-month follow-up, 12-month follow-up) are presented in Table 7. There were no significant changes
found in work engagement following the R2MR training intervention. Specifically, work
engagement scores were associated with non-significant changes at both the 6-month follow-up, d = 0.050 (95% CI [−0.44, 0.53]), and 12-month follow-up, d = −0.134 (95% CI
[−0.52, 0.19]). Workplace stigma attitude scores were associated with small, but significant (p = .031), decreases in workplace stigma attitude scores at post-training, d = −0.287

COGNITIVE BEHAVIOUR THERAPY 

Table 2. R2MR-specific skills training questions.
Post-training

6-month follow-up

12-month follow-up

% (n)

% (n)

% (n)

2.0 (2)
12.2 (12)
22.4 (22)
49.0 (48)
14.3 (14)

5.0 (2)
20.0 (8)
35.0 (14)
30.0 (12)
10.0 (4)

1.7 (1)
21.7 (13)
50.0 (30)
23.3 (14)
3.3 (2)

2 (2.0)
16 (16.3)
34 (34.7)
33 (33.7)
13 (13.3)

1 (2.5)
10 (25.0)
15 (37.5)
13 (32.5)
1 (2.5)

2 (3.3)
17 (28.3)
27 (45.0)
12 (20.0)
2 (3.3)

1.0 (1)
7.1 (7)
35.7 (35)
43.9 (43)
12.2 (12)

2.5 (1)
30.0 (12)
55.0 (22)
10.0 (4)
2.5 (1)

5.0 (3)
36.7 (22)
45.0 (27)
8.3 (5)
5.0 (3)

3.1 (3)
10.2 (10)
41.8 (41)
37.8 (37)
7.1 (7)

5.0 (2)
37.5 (15)
37.5 (15)
17.5 (7)
2.5 (1)

3.3 (2)
40.0 (24)
35.0 (21)
20.0 (12)
1.7 (1)

Post-training

6-month follow-up

12-month follow-up

% (n)

% (n)

% (n)

36.7 (36)
19.4 (19)
31.6 (31)
10.2 (10)
2.0 (2)

30.0 (12)
30.0 (12)
30.0 (12)
1.4 (4)
0.0 (0)

45.0 (27)
20.0 (12)
31.7 (19)
1.7 (1)
1.7 (1)

26.5 (26)
27.6 (27)
30.6 (30)
14.3 (14)
1.0 (1)

27.5 (11)
25.0 (10)
37.5 (15)
10.0 (4)
0.0 (0)

30.0 (18)
31.7 (19)
21.7 (13)
16.7 (10)
0.0 (0)

35.7 (35)
26.5 (26)
27.6 (27)
8.2 (8)
2.0 (2)

20.0 (8)
45.0 (18)
32.5 (13)
2.5 (1)
0.0 (0)

40.0 (24)
30.0 (18)
21.7 (13)
8.3 (5)
0.0 (0)

59.2 (58)
19.4 (19)
18.4 (18)
2.0 (2)
1.0 (1)

65.0 (26)
20.0 (8)
15.0 (6)
0.0 (0)
0.0 (0)

56.7 (34)
23.3 (1)
15.0 (9)
3.3 (2)
1.7 (1)

How helpful did you find the R2MR training?
 Not at all
 A little
Somewhat
 Very
 Extremely
How much do you feel you learned?
 None at all
 A little
Some
 A lot
 A great deal
How much do you remember about R2MR?
 None at all
 A little
Some
 A lot
 A great deal
How well do you feel you can implement the skills
from R2MR?
 Not at all
 A little
Somewhat
 Very
 Extremely

Table 3. R2MR changes from training questions.

Since completing R2MR have you noticed changes
in the better for yourself regarding mental health?
 None at all
 A little
Some
 A lot
 A great deal
Since completing R2MR have you noticed changes
for the better in your leadership with respect to
mental health?
 None at all
 A little
Some
 A lot
 A great deal
Since completing R2MR have you noticed changes
for the better in your colleagues with respect to
mental health?
 None at all
 A little
Some
 A lot
 A great deal
How often have you accessed resources based on
R2MR?
 Not at all
 A little
Somewhat
 Very
 Extremely

9

How often did you find use of the following skills or resources?
Goal setting
 Post-training
6-month follow-up
12-month follow-up
Mental rehearsal/visualization
 Post-training
6-month follow-up
12-month follow-up
Self talk
 Post-training
6-month follow-up
12-month follow-up
Arousal management (Tactical breathing)
 Post-training
6-month follow-up
12-month follow-up
Knowledge about resources and confidence in accessing them
 Post-training
6-month follow-up
12-month follow-up
“Shield, sense, support” (leadership only)
 Post-training
6-month follow-up
12-month follow-up
Mental health continuum model
 Post-training
6-month follow-up
12-month follow-up
The pocket card
 Post-training
6-month follow-up
12-month follow-up
The aide memoire (leadership booklet)
 Post-training
6-month follow-up
12-month follow-up

Table 4. R2MR skills utilization questions.

7.4 (7)
2.5 (1)
5.0 (3)
5.3 (5)
7.5 (3)
8.3 (5)
4.2 (4)
0.0 (0)
6.7 (4)
8.4 (8)
12.5 (5)
10.0 (6)
11.6 (11)
15.0 (6)
20.0 (12)
6.3 (6)
7.5 (3)
20.0 (12)
12.6 (12)
7.5 (3)
11.7 (7)
10.5 (10)
12.5 (5)
13.3 (8)
10.5 (10)
12.5 (5)
8.3 (5)

7.4 (7)
22.5 (9)
18.3 (11)
8.4 (8)
10.0 (4)
11.7 (7)
10.5 (10)
17.5 (7)
18.3 (11)
9.5 (9)
15.0 (6)
18.3 (11)
33.7 (33)
57.5 (23)
36.7 (22)
14.7 (14)
27.5 (11)
21.7 (13)
32.6 (31)
40.0 (16)
55.0 (33)
47.4 (45)
70.0 (28)
73.3 (44)

% (n)

% (n)

6.3 (6)
20.0 (8)
21.7 (13)

Almost never

Never

15.8 (15)
7.5 (3)
6.7 (4)

21.1 (20)
17.5 (7)
10.0 (6)

18.9 (18)
20.0 (8)
20.0 (12)

17.9 (17)
20.0 (8)
15.0 (9)

18.9 (18)
15.0 (6)
23.3 (14)

15.8 (15)
25.0 (10)
13.3 (8)

8.4 (8)
17.5 (7)
11.7 (7)

10.5 (10)
17.5 (7)
13.3 (8)

6.3 (6)
12.5 (5)
11.7 (7)

% (n)

Rarely

20.0 (19)
10.0 (4)
10.0 (6)

24.2 (23)
20.0 (8)
10.0 (6)

26.3 (25)
30.0 (12)
28.3 (17)

26.3 (25)
12.5 (5)
15.0 (9)

36.8 (35)
27.5 (11)
16.7 (10)

34.7 (33)
32.5 (13)
31.7 (19)

18.9 (18)
35.0 (14)
16.7 (10)

27.4 (26)
27.5 (11)
18.3 (11)

28.4 (27)
25.0 (10)
26.7 (16)

% (n)

Sometimes

5.3 (5)
0.0 (0)
1.7 (1)

7.4 (7)
7.5 (3)
6.7 (4)

18.9 (18)
12.5 (5)
10.0 (6)

10.5 (10)
0.0 (0)
11.7 (7)

14.7 (14)
12.5 (5)
8.3 (5)

21.1 (20)
5.0 (2)
3.2 (9)

29.5 (28)
20.0 (8)
30.0 (18)

31.6 (30)
20.0 (8)
23.3 (14)

26.3 (25)
22.5 (9)
21.7 (13)

% (n)

Often

1.1 (1)
0.0 (0)
0.0 (0)

3.2 (3)
2.5 (1)
1.7 (1)

6.3 (6)
2.5 (1)
3.3 (2)

4.2 (4)
2.5 (1)
0.0 (0)

6.3 (6)
12.5 (5)
11.7 (7)

4.2 (4)
5.0 (2)
6.7 (4)

18.9 (18)
15.0 (6)
13.3 (8)

10.5 (10)
2.5 (1)
13.3 (8)

17.9 (17)
15.0 (6)
11.7 (7)

% (n)

Very often

0.0 (0)
0.0 (0)
0.0 (0)

1.1 (1)
0.0 (0)
3.3 (2)

2.1 (2)
0.0 (0)
5.0 (3)

1.1 (1)
0.0 (0)
1.7 (1)

2.1 (2)
2.5 (1)
1.7 (1)

5.3 (5)
2.5 (1)
5.0 (3)

11.6 (11)
2.5 (1)
10.0 (6)

7.4 (7)
2.5 (1)
5.0 (3)

2.5 (7)
2.5 (1)
1.7 (1)

% (n)

Always

10
 R. N. CARLETON ET AL.

COGNITIVE BEHAVIOUR THERAPY 

11

Table 5. R2MR skills helpfulness questions.

How helpful did you find each of the following skills or resources:
Goal setting
 Post-training
6-month follow-up
12-month follow-up
Mental rehearsal/visualization
 Post-training
6-month follow-up
12-month follow-up
Self talk
 Post-training
6-month follow-up
12-month follow-up
Arousal management (Tactical breathing)
 Post-training
6-month follow-up
12-month follow-up
Knowledge about resources and confidence
in accessing them
 Post-training
6-month follow-up
12-month follow-up
“Shield, sense, support” (leadership only)
 Post-training
6-month follow-up
12-month follow-up
Mental health continuum model
 Post-training
6-month follow-up
12-month follow-up
The pocket card
 Post-training
6-month follow-up
12-month follow-up
The aide memoire (leadership booklet)
 Post-training
6-month follow-up
12-month follow-up

Not at all

A little bit

Moderately

Quite a bit

Extremely

n (%)

n (%)

n (%)

n (%)

n (%)

4.2 (4)
15.0 (6)
18.3 (11)

17.9 (17)
20.0 (8)
26.7 (16)

16.8 (16)
25.0 (10
20.0 (12)

47.4 (45)
32.5 (13)
30.0 (18)

13.7 (13)
7.5 (3)
5.0 (3)

7.4 (7)
15.0 (6)
15.0 (9)

12.6 (12)
22.5 (9)
18.3 (11)

20.0 (19)
20.0 (8)
25.0 (15)

43.2 (41)
35.0 (14)
33.3 (20)

16.8 (16)
7.5 (3)
8.3 (5)

5.3 (5)
10.0 (4)
11.7 (7)

12.6 (12)
27.5 (11)
23.3 (14)

23.2 (22)
27.5 (11)
23.3 (14)

40.0 (38)
25.0 (10)
31.7 (19)

18.9 (18)
10.0 (4)
10.0 (6)

4.2 (4)
27.5 (11)
16.7 (10)

22.1 (21)
25.0 (10)
23.3 (14)

28.4 (27)
22.5 (9)
28.3 (17)

33.7 (32)
17.5 (7)
25.0 (15)

11.6 (11)
7.5 (3)
6.7 (4)

7.4 (7)
17.5 (7)
16.7 (10)

13.7 (13)
15.0 (6)
26.7 (16)

33.7 (32)
45.0 (18)
38.3 (23)

31.6 (30
20.0 (8)
15.0 (9)

13.7 (13)
2.5 (1)
3.3 (2)

29.5 (28)
47.5 (19)
40.0 (24)

18.9 (18)
12.5 (5)
25.0 (15)

21.1 (20)
27.5 (11)
20.0 (12)

22.1 (21)
10.0 (4)
11.7 (7)

8.4 (8)
2.5 (1)
3.3 (2)

8.4 (8)
17.5 (7)
11.7 (7)

16.8 (16)
17.5 (7)
26.7 (16)

28.4 (27)
37.5 (15)
35.0 (21)

33.7 (32)
17.5 (7)
18.3 (11)

12.6 (12)
10.0 (4)
8.3 (5)

12.6 (12)
37.5 (15)
41.7 (25)

27.4 (26)
20.0 (8)
23.3 (14)

20.0 (19)
15.0 (6)
20.0 (12)

26.3 (25)
20.0 (8)
8.3 (5)

13.7 (13)
7.5 (3)
6.7 (4)

38.9 (37)
50.0 (20)
53.3 (32)

18.9 (18)
22.5 (9)
18.3 (11)

18.9 (18)
25.0 (10)
18.3 (11)

17.9 (17)
2.5 (1)
8.3 (5)

5.3 (5)
0.0 (0)
1.7 (1)

(95% CI [−0.50, −0.14]); however, the changes were non-significant at both the 6-month
follow-up, d = −0.140 (95% CI [−0.37, −0.00]), and the 12-month follow-up, d = −0.190
(95% CI [−0.46, 0.01]).
Open-ended participant responses
Immediately following the R2MR training, most participants reported the following themes
when asked questions from the R2MR-Specific questionnaire; when asked, “What did you
find most helpful?”: improved understanding of mental health (in themselves and coworkers), the use of real-world examples (i.e. video testimonials and instructor stories),
group discussions, how to communicate about mental health, how to access resources for
mental health, how to reduce stigma, and how to use the different R2MR tools (e.g. the
Mental Health Continuum Model). When asked to provide examples about what participants remembered about R2MR, most participants reported the Mental Health Continuum

I understand how mental health problems present in the workplace
 Post-training
6-month follow-up
12-month follow-up
I plan to seek help for my mental health problems when needed
 Post-training
6-month follow-up
12-month follow-up
When I am concerned I ask my colleagues how they are doing
 Post-training
6-month follow-up
12-month follow-up
I talk about mental health issues as freely as physical health issues
 Post-training
6-month follow-up
12-month follow-up
I understand management practices that promote the performance and well being of all
employees
 Post-training
6-month follow-up
12-month follow-up

Table 6. Mental health stigma questions.

3.2 (3)
5.0 (2)
1.7 (1)
4.2 (4)
10.0 (4)
3.3 (2)
3.2 (3)
0.0 (0)
1.7 (1)
20.0 (19)
10.0 (4)
28.3 (17)
7.4 (7)
17.5 (7)
13.3 (8)

0.0 (0)
0.0 (0)
3.3 (2)
1.1 (1)
5.0 (2)
1.7 (1)
8.4 (8)
10.0 (4)
6.7 (4)
2.1 (2)
5.0 (2)
6.7 (4)

n (%)

n (%)
1.1 (1)
0.0 (0)
0.0 (0)

Disagree

Strongly disagree

24.2 (23)
30.0 (12)
28.3 (17)

18.9 (18)
25.0 (10)
25.0 (15)

9.5 (9)
15.0 (6)
11.7 (7)

17.9 (17)
12.5 (5)
20.0 (12)

18.9 (18)
15.0 (6)
8.3 (5)

n (%)

Neither agree nor disagree

53.7 (51)
42.5 (17)
43.3 (26)

44.2 (42)
40.0 (16)
31.7 (19

62.1 (59)
57.5 (23)
68.3 (41)

58.9 (56)
57.5 (23)
55.0 (33)

65.3 (62)
72.5 (29)
78.3 (47)

Agree

12.6 (12)
5.0 (2)
8.3 (5)

8.4 (8)
15.0 (6)
8.3 (5)

24.2 (23)
22.5 (9)
16.7 (10)

18.9 (18)
20.0 (8)
18.3 (11)

11.6 (11)
7.5 (3)
11.7 (7)

n (%)

Strongly agree

12
 R. N. CARLETON ET AL.

COGNITIVE BEHAVIOUR THERAPY 

13

Table 7. Multilevel models for dependent measures from baseline to 12-month follow-up.
Fixed effects
Predictor
DASS-21 depression
 Intercept
12-month
6-month
DASS-21 anxiety
 Intercept
12-month
6-month
DASS-21 stress
 Intercept
12-month
6-month
MAKS
 Intercept
12-month
6-month
PCL-5
 Intercept
12-month
6-month
AUDIT
 Intercept
12-month
6-month
UWES-9
 Intercept
12-month
6-month
BRS
 Intercept
12-month
6-month
OMSWA
 Intercept
12-month
6-month
 Post-training

Random Effects

b

SE

Variance

3.84**
0.17
0.01

0.41
0.21
0.65

10.56***

9.79***

2.62**
−0.63
0.07

0.24
0.52
0.66

4.32

5.27**

5.75**
−0.54
−0.24

0.35
0.76
0.92

9.45

9.43

46.50**
−0.06
0.16

0.36
0.73
0.85

5.74

10.08**

13.84**
0.97
1.86

1.13
2.70
2.49

44.39

155.49**

4.17**
−0.19
−0.14

0.24
0.57
0.39

1.38

6.95**

32.67**
−1.09
0.41

0.73
1.93
1.80

34.63

32.47

2.94**
0.03
−0.02

0.03
0.73
0.08

0.07

0.02

2.03**
−0.09
−0.07
−0.14

0.04
0.09
0.07
0.06

0.07

0.15**

SE

Note: SE = standard error; DASS-21 = Depression Anxiety Stress Scales 21-item; PCL-5 = Posttraumatic Stress Disorder Checklist-5; AUDIT = Alcohol Use Disorders Identification Test; UWES-9 = Utrecht Work Engagement Scale 9-item;
BRS = Brief Resilience Scale; MAKS = Mental Health Knowledge Schedule; OMSWA = Open Minds Survey of Workplace
Attitudes. Random effects were estimated using a variance components covariance matrix.
*p ≤ .05.; **p ≤ .01.; ***p ≤ .001.

Model, as well as visual examples. When asked, “What skills will be the most challenging
to implement and why”, most participants reported the following themes: acceptance of
when you need help, how to approach co-workers, and getting others to take mental health
seriously (i.e. peer support, reducing stigma). When asked, “What skills will be the easiest
to implement and why?”, most participants reported the following themes: active listening (i.e. empathetic listening), being more approachable and supportive for co-workers,
self-awareness, tactical breathing, and goal setting.
At the 6-month follow-up, when asked, “What did you find most helpful?” most participants reported the following themes: using the Mental Health Continuum, increased
knowledge and awareness about mental health, and coping strategies. A few participants
specifically reported that following R2MR they were able to recognize the severity of their

14

 R. N. CARLETON ET AL.

own mental health issues and sought out help. When asked to provide examples about
what participants remembered about R2MR, most participants reported the Mental Health
Continuum, the importance of addressing mental health stigma, and how to be empathetic
and supportive toward co-workers. A few participants specifically reported being unable to
remember any examples of what they learned from the R2MR training. When asked, “What
skills will be the most challenging to implement and why?”, most participants reported
similar themes to the post-training questionnaire (i.e. how to approach co-workers, particularly for fear of a defensive reaction or denial of the issue), as well as self-care in times
of heightened stress and increased deadlines. When asked, “What skills will be the easiest
to implement and why?” most participants reported similar themes to the post-training
questionnaire (i.e. empathetic listening, being more approachable and supportive toward
co-workers, self-awareness, tactical breathing, and goal setting).
At the 12-month follow-up, when asked, “What did you find most helpful?” most participants reported the following themes: increased awareness, understanding, and communication about mental health, the Mental Health Continuum Model, learning how to
use a common language to discuss mental health concerns, and how to reduce stigma
surrounding mental health issues. A few participants reported being unable to identify
anything as helpful, commenting that the material was too basic for the intended audience.
When asked to provide examples about what participants remembered about R2MR, most
participants reported the Mental Health Continuum, tactical breathing techniques, selftalk, and goal-setting. When asked, “What skills will be the most challenging to implement
and why?”, most participants reported similar themes to the previous questionnaires, and
noted that they have difficulty balancing work expectations with mental health awareness
(i.e. being able to take a mental health day, or feeling judged by other co-workers when
unable to meet deadlines due to mental health concerns), as well as difficulty implementing
the goal setting skill. When asked, “What skills will be the easiest to implement and why?”
most participants reported similar themes to the previous questionnaires. Some participants
emphasized that even a year later the training had facilitated more open dialogs and easier
communication thanks to the mental health continuum.

Discussion
The current study assessed the impact of the MHCC R2MR training at pre-training,
post-training, 6-month follow-up, and 12-month follow-up on several factors in a sample of
Canadian municipal police officers. We assessed for quantified changes based on responses
to several validated questionnaires. We also thematically assessed open-ended responses
to the training questions.
The multilevel analytic results suggested there were no statistically significant changes
in symptoms of depression, anxiety, stress, posttraumatic stress, and alcohol use, at any
follow-up time point, following the training intervention. There were very small increases in
depression and PTSD at 6 and 12 months, small increases in anxiety at 6 months, and small
decreases in anxiety at 12 months, as well as small decreases in reported stress and reported
alcohol use at both 6 and 12 months. The absence of significant changes in symptoms of
depression, anxiety, stress, alcohol use, or PTSD are not necessarily surprising, given that
the R2MR program is not intended as a treatment, but as a tool for providing education,
increasing resilience, and reducing stigma.

COGNITIVE BEHAVIOUR THERAPY 

15

The multilevel analytic results also suggested that no statistically significant changes
in mental health knowledge, resilience, or workplace engagement occurred following the
R2MR program. There were nonetheless small increases in mental health knowledge at 6
months and in resilience scores at 12 months. There were small increases in work engagement at 6 months, but then decreases at 12 months. Only mental health stigma was assessed
at all four time points. The results of the multilevel analyses suggested significant small
reductions immediately post-training, and non-significant small reductions at 6 and 12
months. The results suggest a single session of R2MR training may reduce stigma over time,
but sustained reductions may require refresher training sessions.
In their own open-ended responses, participants implied the program had a greater
impact than indicated by changes in the questionnaires. Immediately after the training
most participants reported improved understanding of mental health, stronger skills for
discussing mental health, improved clarity regarding mental health resources, lower stigma,
and appreciation of the R2MR tools, particularly the Mental Health Continuum model.
Perhaps importantly, the post-training assessment would not have afforded participants
much time to engage with the newly presented materials and skills, so those assessments
should be evaluated with caution. At the 6- and 12-month follow-ups, most participants
reported the same improvements and foci, with some attributing improved interactions with
their own mental health as a direct function of the training; however, some were unable to
report recalling any skills specific to the training. At 12 months, some participants explicitly
stated that in retrospect the material was too basic for their needs relative to their extant
education; nevertheless, at one year later most still reported recalling key aspects of R2MR
(i.e. Mental Health Continuum, tactical breathing techniques, self-talk, and goal-setting). At
all three time points the open-ended responses to perceived difficulties focused on stigma
at the individual, community, and structural levels.
Perceived helpfulness, learning, content recall, and skill confidence appeared to decline
from post-training to the 12-month follow-up, suggesting that refresher sessions may be
important for retention. Participants’ perceived changes in themselves, their leadership, and
their colleagues were less consistent, but generally indicated similarly small changes. More
than half of participants reported using Goal Setting, Mental Rehearsal/Visualization, and
Self Talk, Sometimes, Often, Very Often, or Always, at all three time points; however, without
a pre-training assessment there is no way to measure the specific impact of R2MR on those
skills. Other skill use (i.e. Mental Health Continuum, Tactical Breathing, Knowledge about
Resources and Confidence in Accessing them, Shield, Sense, Support, the Pocket Card, the
Aide Memoire) occurred more frequently post-training than at either follow-up. Among
participants’ who used the skills, more than half perceived the helpfulness of Goal Setting,
Mental Rehearsal/Visualization, Self Talk, Tactical Breathing, Knowledge about Resources
and Confidence in Accessing them, and the Mental Health Continuum, as Moderately, Quite
a bit, or Extremely at all three time points. Other skills (i.e. Shield, Sense, Support, the Pocket
Card, the Aide Memoire) were similarly perceived as helpful at post-training, but more
than half of participants described them as Not at all or A little bit helpful by the 12-month
follow-up. Overall, participants reported that psychoeducation, using the Mental Health
Continuum, and skills for communicating about mental health were all particularly beneficial; however, participants reported concerns that they would have difficulty (1) accepting
their own need for help; (2) asking for help for fear of judgment or negative consequences;
and (3) being as helpful as they would want to be for colleagues with mental health concerns.

16

 R. N. CARLETON ET AL.

The absence of larger changes in the quantitative measures may indicate the training as
delivered and used after delivery would yet benefit from additional revision; however, based
on the open-ended responses, there are also several other possibilities. The length of training
and the absence of training refresher courses may have been insufficient to produce larger
changes (cf. Anderson, Gaetz, Statz, & Kin, 2012). The quantitative measures may have
been insufficiently sensitive to change. There was an 11.5% increase in the employee and
family assistance program (EFAP) costs during the calendar year that followed the R2MR
training, as well as a further increase of 4.8% in the subsequent calendar year, which implies
the training might have increased use; however, there is no way to clarify a specific causal
relationship or delineate the impact of inflation and the data are not available to compare
the increases to prior years. There may also have been an unaccounted for expectancy effect.
There may have been a ceiling effect, minimizing room for further improvement, and driven
by practices specific to the police agency (e.g. leadership, support systems, training, social
norms), growing public education and awareness regarding mental health, and positive
results of broad anti-stigma campaigns. In any case, more research appears necessary to
understand how best to continue supporting mental health.

Limitations
The current study has several limitations that offer direction for future research. First, our
responses were based on anonymous self-report, which always allows for biased responding,
erroneous responding, and missing data. Concerns about stigma and possible identifiability
may also have impacted participant reports (e.g. Halpern, Gurevich, Schwartz, & Brazeau,
2009; Henderson, Van Hasselt, Leduc, & Couwels, 2016; Karaffa & Koch, 2016). Relatedly,
in the absence of diagnostic interviews assessments of symptoms and skills, as well as absent
assessment of objective behavioral indicators, the current results may not replicate and may
not reflect the full impact of the training program. Future researchers should consider using
diagnostic assessment interviews instead of relying entirely on self-report screening tools.
Second, participants were not assessed for how often they reviewed the R2MR materials, nor
were they required to review such materials, which means knowledge and skill acquisition
was focused on a single session of training. There is growing meta-analytic evidence that
brief interventions can produce behavior changes with small to moderate effect sizes (e.g.
(Aveyard, Begh, Parsons, & West, 2012; Sagherian, Huedo-Medina, Pellowski, Eaton, &
Johnson, 2016; Sullivan, Tetrault, Braithwaite, Turner, & Fiellin, 2011) and similar evidence
specifically for resilience (Leppin et al., 2014; Robertson et al., 2015); however, programs
that are longer and that are directed toward people with specific problems or issues can be
expected to produce larger outcomes. There is also evidence that skills are enhanced when
programs involve repeated skill review and ongoing training (Anderson et al., 2012). Future
researchers should consider engaging training refreshers to assess for the incremental impact
of additional training.
Third, only mental health stigma was assessed at all four time points because the training
was not expected to produce an imminent change in symptoms; nevertheless, given the
immediacy of impact in mental health stigma, future researchers may want to assess for
similar immediate improvements in symptoms as a result of training. Fourth, the quantified results based on the self-report measures appear to contrast the open-ended responses
provided by participants; specifically, the relatively small quantified changes do not reflect

COGNITIVE BEHAVIOUR THERAPY 

17

participant comments that the program was perceived as beneficial. The disparity may be the
result of gaps in measurement, participant perceptual biases, or implicit demand characteristics. Future researchers should carefully consider the assessment tools for evaluating program impact and, where possible, include quantitative, qualitative, and objective behavioral
measures, such as changes in EFAP usage. Fifth, participants were trained by trainers using
the MHCC adaptation of R2MR; accordingly, there is no way to assess the impact of the
modification, whether the results would be robust across training from other organizations
(e.g. the DND Canadian Armed Forces), across other populations (e.g. other PSP, military),
or with other variants of R2MR (e.g. adaptations of R2MR for PSP by organizations other
than MHCC, such as the recent DND Canadian Armed Forces adaptation for all public
safety personnel of the most recent version of R2MR). Future researchers should assess for
the impact of such differences on outcomes associated with R2MR training. Sixth, changes
to stigma may be dependent on cultural changes within the policing context; as such,
measurable indicators of change (e.g. increased early help-seeking and reduced long-term
health costs) can reasonably be expected to require sustained dedicated effort over several
years with demonstrable results being evidenced slowly over time. If such expectations are
true, investments in additional interventions and longitudinal research will be required to
create and evaluate such changes.

Conclusions
The current results suggest that the R2MR program adapted by MHCC and delivered to
a municipal police force was able to produce effects generally consistent with previous
research results for single session resiliency interventions. Participants were able to recall
R2MR components 12 months after training. On the self-report measures, participants did
not report significant changes in mental health symptoms, resilience, or work engagement,
but did report small increases in mental health knowledge and small temporary decreases
in stigma; in contrast, the open-ended participant responses indicated that most found the
training helpful for increasing mental health awareness, understanding, and communication skills, as well as helping to reduce stigma by correcting erroneous beliefs about mental
health and perceptions of consequences for accessing mental health care (Karaffa & Koch,
2016), which may increase help seeking. In this context, researching the R2MR program
delivered by MHCC, which is a brief intervention using cognitive behavioral therapy style
skills developed for a broad audience, has provided important results; nevertheless, PSP
researchers and trainers should explore the contemporary DND adaptation of R2MR, as
well as continue pursuing opportunities to improve evidence-informed cognitive behavioral
therapy style training programs, intervention programs, and implementation processes in
support of PSP resilience and mental health.

Acknowledgments
The research could not have been conducted without the tremendous support of the Regina Police
Service, particularly from Alissa Sullivan, the EFAP Health & Safety Coordinator.

Disclosure statement
No potential conflict of interest was reported by the authors.

18

 R. N. CARLETON ET AL.

Funding
R. N. Carleton’s research is supported in part by the Canadian Institutes of Health Research (CIHR)
through a New Investigator Award [grant number FRN: 285489].

ORCID
R. Nicholas Carleton
http://orcid.org/0000-0002-6083-8935
http://orcid.org/0000-0002-6063-5230
Julia E. Mason
http://orcid.org/0000-0003-0458-4616
Kadie Hozempa

References
Adams, G. A., & Buck, J. (2010). Social stressors and strain among police officers: It’s not just the bad
guys. Criminal Justice and Behavior, 37, 1030–1040. https://doi.org/10.1177/0093854810374282
Anderson, G. S., Gaetz, M., Statz, C., & Kin, B. (2012). CPR skill retention of first aid attendants
within the workplace. Prehospital and Disaster Medicine, 27, 312–318. https://doi.org/10.1017/
S1049023X1200088X
Anderson, G. S., Litzenberger, R., & Plecas, D. (2002). Physical evidence of police officer stress.
Policing: An International Journal of Police Strategies & Management, 25, 399–420. https://doi.
org/10.1108/13639510210429437
Anderson, G. S., Vaughan, A. D., & Mills, S. (2017). Building personal resilience in paramedic
students. Journal of Community Safety & Well-Being, 2, 51–54.
Anshel, M. H. (2000). A conceptual model and implications for coping with stressful events in police
work. Criminal Justice and Behavior, 27, 375–400. https://doi.org/10.1177/0093854800027003006
Antony, M. M., Bieling, P. J., Cox, B. J., Enns, M. W., & Swinson, R. P. (1998). Psychometric properties
of the 42-item and 21-item versions of the Depression Anxiety Stress Scales in clinical groups
and a community sample. Psychological Assessment, 10, 176–181. https://doi.org/10.1037/10403590.10.2.176
Aveyard, P., Begh, R., Parsons, A., & West, R. (2012). Brief opportunistic smoking cessation
interventions: A systematic review and meta-analysis to compare advice to quit and offer of
assistance. Addiction, 107, 1066–1073. https://doi.org/10.1111/j.1360-0443.2011.03770.x
Blevins, C. A., Weathers, F. W., Davis, M. T., Witte, T. K., & Domino, J. L. (2015). The posttraumatic
stress disorder checklist for DSM-5 (PCL-5): Development and initial psychometric evaluation.
Journal of Traumatic Stress, 28, 489–498. https://doi.org/10.1002/jts.22059.
Bohn, M. J., Babor, T. F., & Kranzler, H. R. (1995). The alcohol use disorders identification test
(AUDIT): Validation of a screening instrument for use in medical settings. Journal of Studies on
Alcohol, 56, 423–432. https://doi.org/10.15288/jsa.1995.56.423 7674678
Bovin, M. J., Marx, B. P., Weathers, F. W., Gallagher, M. W., Rodriguez, P., Schnurr, P. P., & Keane, T.
M. (2016). Psychometric properties of the PTSD checklist for diagnostic and statistical manual
of mental disorders-fifth edition (PCL-5) in veterans. Psychol Assess, 28, 1379–1391. https://doi.
org/10.1037/pas0000254.
Brown, J., & Campbell, E. (1994). Stress and policing: Sources and strategies. Chichester: Wiley.
Carleton, R. N., Afifi, T. O., Turner, S., Taillieu, T., Duranceau, S., LeBouthillier, D. M., & Asmundson,
G. J. G. (2018). Mental disorder symptoms among public safety personnel. The Canadian Journal
of Psychiatry, 63, 54–64. https://doi.org/10.1177/0706743717723825
Carleton, R. N., Afifi, T. O., Turner, S., Taillieu, T., El-Gabalawy, R., Sareen, J., & Asmundson, G. J.
G. (2017). Chronic pain among public safety personnel in Canada. Canadian Journal of Pain, 1,
237–246. https://doi.org/10.1080/24740527.2017.1410431
Carleton, R. N., Afifi, T. O., Turner, S., Taillieu, T., LeBouthillier, D. M., Duranceau, S., … Asmundson,
G. J. G. (in press). Suicidal ideation, plans, and attempts among public safety personnel in Canada.
Canadian Psychology/Psychologie canadienne. https://doi.org/10.1037/cap0000136

COGNITIVE BEHAVIOUR THERAPY 

19

Clara, I., Cox, B. J., & Enns, M. W. (2001). Confirmatory Factor analysis of the depression–anxiety–
stress scales in depressed and anxious patients. Journal of Psychopathology and Behavioral
Assessment, 23, 61–67.
Collins, P. A., & Gibbs, A. C. (2003). Stress in police officers: A study of the origins, prevalence
and severity of stress-related symptoms within a county police force. Occupational Medicine, 53,
256–264. https://doi.org/10.1093/occmed/kqg061
Davey, J. D., Obst, P. L., & Sheehan, M. C. (2001). It goes with the job: Officers’ insights into the impact
of stress and culture on alcohol consumption within the policing occupation. Drugs-Education
Prevention and Policy, 8, 141–149. https://doi.org/10.1080/096876301300101889
Evans-Lacko, S., Little, K., Meltzer, H., Rose, D., Rhydderch, D., Henderson, C., & Thornicroft, G.
(2010). Development and psychometric properties of the mental health knowledge schedule. The
Canadian Journal of Psychiatry, 55, 440–448. https://doi.org/10.1177/070674371005500707
Gache, P., Michaud, P., Landry, U., Accietto, C., Arfaoui, S., Wenger, O., & Daeppen, J. B. (2005).
The alcohol use disorders identification test (AUDIT) as a screening tool for excessive drinking
in primary care: Reliability and validity of a French version. Alcoholism-Clinical and Experimental
Research, 29, 2001–2007. https://doi.org/10.1097/01.alc.0000187034.58955.64
Garcia, L., Nesbary, D. K., & Gu, J.. (2004). Perceptual variations of stressors among police officers
during an era of decreasing crime. Journal of Contemporary Criminal Justice, 20, 33–50. https://
doi.org/10.1177/1043986203262300
Gershon, R. R. M., Barocas, B., Canton, A. N., Li, X., & Vlahov, D.. (2009). Mental, physical, and
behavioral outcomes associated with perceived work stress in police officers. Criminal Justice and
Behavior, 36, 275–289. https://doi.org/10.1177/0093854808330015
Gloster, A. T., Rhoades, H. M., Novy, D., Klotsche, J., Senior, A., Kunik, M., & Stanley, M. A. (2008).
Psychometric properties of the depression anxiety and stress scale-21 in older primary care patients.
Journal of Affective Disorders, 110, 248–259. https://doi.org/10.1016/j.jad.2008.01.023
Hakan Can, S., & Hendy, H. M. (2014). Police stressors, negative outcomes associated with them and
coping mechanisms that may reduce these associations. Police Journal, 87(3), 167–177. https://
doi.org/10.1350/pojo.2014.87.3.676
Halpern, J., Gurevich, M., Schwartz, B., & Brazeau, P. (2009). What makes an incident critical for
ambulance workers? Emotional outcomes and implications for intervention. Work and Stress, 23,
173–189. https://doi.org/10.1080/02678370903057317
Hansson, L., & Markstrom, U. (2014). The effectiveness of an anti-stigma intervention in a basic
police officer training programme: A controlled study. BMC Psychiatry, 14, 55. https://doi.
org/10.1186/1471-244x-14-55
Henderson, S. N., Van Hasselt, V. B., Leduc, T. J., & Couwels, J. (2016). Firefighter suicide:
Understanding cultural challenges for mental health professionals. Professional Psychology-Research
and Practice, 47, 224–230. https://doi.org/10.1037/pro0000072
Houdmont, J., Randall, R., Kerr, R., & Addley, K. (2013). Psychosocial risk assessment in organizations:
Concurrent validity of the brief version of the management standards indicator tool. Work & Stress,
27, 403–412. https://doi.org/10.1080/02678373.2013.843607
Karaffa, K. M., & Koch, J. M. (2016). Stigma, pluralistic ignorance, and attitudes toward seeking
mental health services among police officers. Criminal Justice and Behavior, 43, 759–777. https://
doi.org/10.1177/0093854815613103
Leontjevas, R., de Beek, W. O., Lataster, J., & Jacobs, N. (2014). Resilience to affective disorders: A
comparative validation of two resilience scales. Journal of Affective Disorders, 168, 262–268. https://
doi.org/10.1016/j.jad.2014.07.010.
Leppin, A. L., Bora, P. R., Tilburt, J. C., Gionfriddo, M. R., Zeballos-Palacios, C., Dulohery, M. M., &
Montori, V. M. (2014). The efficacy of resiliency training programs: A systematic review and metaanalysis of randomized trials. PLoS ONE, 9, e111420. https://doi.org/10.1371/journal.pone.0111420
Lovibond, P. F., & Lovibond, S. H. (1995). The structure of negative emotional states: Comparison
of the depression anxiety stress scales (DASS) with the beck depression and anxiety inventories.
Behaviour Research and Therapy, 33, 335–343.

20

 R. N. CARLETON ET AL.

Maguen, S., Metzler, T. J., McCaslin, S. E., Inslicht, S. S., Henn-Haase, C., Neylan, T. C., & Marmar,
C. R. (2009). Routine work environment stress and PTSD symptoms in police officers. Journal of
Nervous and Mental Disease, 197, 754. https://doi.org/10.1097/NMD.0b013e3181b975f8 19829204.
Mental Health Commission of Canada. (2017). Road to mental readiness. Retrieved from http://
www.r2mr.ca/road-mental-readiness#CentralComponents
Millar, A. (2002). Beyond resolution of presenting issues: Clients’ experiences of an in-house police
counselling service. Counselling and Psychotherapy Research, 2, 159–166. https://doi.org/10.1080
/14733140212331384785
Morash, M., Kwak, D. H., & Haarr, R. (2006). Gender differences in the predictors of police stress.
Policing: An International Journal of Police Strategies & Management, 29, 541–563. https://doi.
org/10.1108/13639510610684755
National Defence and the Canadian Armed Forces. (2017). Road to Mental Readiness (R2MR).
Retrieved from http://www.forces.gc.ca/en/caf-community-health-services-r2mr/index.page
Oliphant, R. (Chair) (2016). Healthy minds, safe communities: Supporting our public safety officers
through a national strategy for operational stress injuries. Canada: Standing Committee on Public
Safety and National Security. Retrieved from http://www.parl.gc.ca/HousePublications/Publication.
aspx?DocId=8457704&Language=E
Papazoglou, K., & Andersen, J. P. (2014). A guide to utilizing police training as a tool to promote
resilience and improve health outcomes among police officers. Traumatology: An International
Journal, 20, 103. https://doi.org/10.1037/h0099394
Pasillas, R. M., Follette, V. M., Perumean-Chaney, S. E. (2006). Occupational stress and psychological
functioning in law enforcement officers. Journal of Police and Criminal Psychology, 21, 41–53.
https://doi.org/10.1007/BF02849501
Piccinelli, M., Tessari, E., Bortolomasi, M., Piasere, O., Semenzin, M., Garzotto, N., & Tansella, M.
(1997). Efficacy of the alcohol use disorders identification test as a screening tool for hazardous
alcohol intake and related disorders in primary care: A validity study. BMJ, 314, 420–420. https://
doi.org/10.1136/bmj.314.7078.420
Robertson, I. T., & Cooper, C. L. (2013). Resilience. Stress and Health: Journal of the International
Society for the Investigation of Stress, 29, 175–176. https://doi.org/10.1002/smi.2512
Robertson, I. T., Cooper, C. L., Sarkar, M., & Curran, T. (2015). Resilience training in the workplace
from 2003 to 2014: A systematic review. Journal of Occupational and Organizational Psychology,
88, 533–562. https://doi.org/10.1111/joop.12120
Rodríguez-Rey, R., Alonso-Tapia, J., & Hernansaiz-Garrido, H. (2016). Reliability and validity of the
brief resilience scale (BRS) spanish version. Psychological Assessment, 28(5), e101–e110. https://
doi.org/10.1037/pas0000191.
Rose, T., & Unnithan, P. (2015). In or out of the group? Police subculture and occupational stress.
Policing: An International Journal of Police Strategies & Management, 38, 279–294. https://doi.
org/10.1108/PIJPSM-10-2014-0111
Sagherian, M. J., Huedo-Medina, T. B., Pellowski, J. A., Eaton, L. A., & Johnson, B. T. (2016). Singlesession behavioral interventions for sexual risk reduction: A meta-analysis. Annals of Behavioral
Medicine, 50, 920–934. https://doi.org/10.1007/s12160-016-9818-4
Saunders, J. B., Aasland, O. G., Babor, T. F., Delafuente, J. R., & Grant, M. (1993). Development
of the alcohol-use disorders identification test (Audit)—who collaborative project on early
detection of persons with harmful alcohol-consumption. 2. Addiction, 88, 791–804. https://doi.
org/10.1111/J.1360-0443.1993.Tb02093.X
Schaufeli, W., Bakker, A. B., & Salanova, M. (2006). The measurement of work engagement with
a short questionnaire: A cross-national study. Educational and Psychological Measurement, 66,
701–716. https://doi.org/10.1177/0013164405282471
Seppala, P., Mauno, S., Feldt, T., Hakanen, J., Kinnunen, U., Tolvanen, A., & Schaufeli, W. (2009). The
construct validity of the utrecht work engagement scale: Multisample and longitudinal evidence.
Journal of Happiness Studies, 10, 459. https://doi.org/10.1007/s10902-008-9100-y
Smith, B. W., Dalen, J., Wiggins, K., Tooley, E., Christopher, P., & Bernard, J. (2008). The brief resilience
scale: Assessing the ability to bounce back. International Journal of Behavioral Medicine, 15, 194–
200. https://doi.org/10.1090/10705500802222972

COGNITIVE BEHAVIOUR THERAPY 

21

Stuart, H., Chen, S. P., Christie, R., Dobson, K., Kirsh, B., Knaak, S., & Whitley, R. (2014). Opening
minds in Canada: Targeting change. The Canadian Journal of Psychiatry, 59, 13–18. https://doi.or
g/10.1177/070674371405901S05
Sullivan, L. E., Tetrault, J. M., Braithwaite, R. S., Turner, B. J., & Fiellin, D. A. (2011). A meta-analysis
of the efficacy of nonphysician brief interventions for unhealthy alcohol use: Implications for the
patient-centered medical home. The American Journal on Addictions, 20, 343–356. https://doi.
org/10.1111/j.1521-0391.2011.00143.x
Szeto, A. C. H., & Adair, M. (2016). Paramedicine and the R2MR—road to mental readiness. Canadian
Paramedicine, 39, 18–21.
Szeto, A. C. H., Luong, D., & Dobson, K. S. (2013). Does labelling matter?: An examination of attitudes
and perceptions of labels for mental disorders. Social Psychiatry and Psychiatric Epidemiology, 48,
659–671. https://doi.org/10.1007/s00127-012-0532-7
van der Velden, P. G., Kleber, R. J., Grievink, L., & Yzermans, J. C. (2010). Confrontations with
aggression and mental health problems in police officers: The role of organizational stressors,
life-events and previous mental health problems. Psychological Trauma: Theory, Research, Practice,
and Policy, 2, 135. https://doi.org/10.1037/a0019158
Volk, R. J., Steinbauer, J. R., Cantor, S. B., & Holzer, C. E. (1997). The alcohol use disorders identification
test (AUDIT) as a screen for at-risk drinking in primary care patients of different racial/ethnic
backgrounds. Addiction, 92, 197–206. https://doi.org/10.1111/j.1360-0443.1997.tb03652.x
Wang, Z., Inslicht, S. S., Metzler, T. J., Henn-Haase, C., McCaslin, S. E., Tong, H., … Marmar, C. R.
(2010). A prospective study of predictors of depression symptoms in police. Psychiatry Research,
175, 211–216. https://doi.org/10.1016/j.psychres.2008.11.010 20044144
Waters, J. A., & Ussery, W. (2007). Police stress: History, contributing factors, symptoms, and
interventions. Policing: An International Journal of Police Strategies & Management, 30, 169–188.
https://doi.org/10.1108/13639510710753199
Weathers, F. W., Litz, B. T., Keane, T. M., Palmieri, P. A., Marx, B. P., & Schnurr, P. P. (2013). The
PTSD checklist for DSM-5 (PCL-5). Scale available from the National Center for PTSD. Retrieved
from www.ptsd.va.gov
Wester, S. R., Arndt, D., Sedivy, S. K., & Arndt, L. (2010). Male police officers and stigma associated
with counseling: The role of anticipated risks, anticipated benefits and gender role conflict.
Psychology of Men & Masculinity, 11, 286–302. https://doi.org/10.1037/a0019108