Research Using In Vivo Simulation of Meta-

Research Using In Vivo Simulation of Meta-
Research Using In Vivo Simulation of MetaOrganizational Shared Decision Making (SDM)
Task 3: Testing the Shared Decision Making Framework in Vivo
Dr. L. Lemyre
University of Ottawa
Scientific Authority:
Paul Chouinard
DRDC Centre for Security Science
The scientific or technical validity of this Contract Report is entirely the responsibility of the Contractor
and the contents do not necessarily have the approval or endorsement of Defence R&D Canada.
Defence R&D Canada – Centre for Security Science
DRDC CSS CR 2011-32
December 2011
Research Using In Vivo Simulation of MetaOrganizational Shared Decision Making (SDM)
Task 3: Testing the Shared Decision Making Framework in Vivo
Produced by:
Dr. L. Lemyre
University of Ottawa
The following GAP-Santé team members contributed to this draft:
Celine Pinsent, PhD,
Marie-Pierre Lalande-Markon, B.Sc.,
Paul Boutette, MBA,
Stacey Gibson, B.A.,
Wayne Corneil, DSc.,
Cecilia Lemus, Eng., MSc.,
Jo Riding, B.A.A.,
Stephanie Blust, B.A.,
David Riding, B.A.,
Hilary M. Kitchener, B.A.
Colleen Johnson, B.A.,
We also acknowledge the valuable contribution of Professor Gilles Paquet.
Scientific authority:
Paul Chouinard
DRDC Centre for Security Science
The scientific or technical validity of this Contract Report is entirely the responsibility of the
Contractor and the contents do not necessarily have the approval or endorsement of Defence Research
and Development Canada.
Defence R&D Canada – CSS
Contractor Report
DRDC CSS CR 2011-32
December 2011
Principal Author
Dr. Louise Lemyre
Professor, Head of Gap-Santé research team
Approved by
Original signed by Paul Chouinard
Paul Chouinard
Operations Research Team Leader, DRDC
Approved for release by
Original signed by Mark Williamson
Mark Williamson
DRDC Centre for Security Science, DDG DRP Chair
© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence 2011,
© Sa Majesté la Reine (en droit du Canada), telle que représentée par le ministre de la Défense nationale 2011,
Abstract ……..
This report documents the completion of Task 3 of the work stream “Research Using in Vivo
Simulation of Meta-Organizational Shared Decision Making (SDM)”, one component of the
Technology Innovation Fund (TIF) program on Meta-organizational Collaboration that has been
designed to assist in understanding challenges faced by the Canadian Forces (CF). The objective
of the stream is to conduct basic research into shared decision making through the analysis of
case studies, exercises and simulations. Task 3 involved the development and testing of the
shared decision making framework in vivo. The research at this stage is to demonstrate that the
model when implemented in vivo can produce improvements in problem solving processes and
outcomes such as better quality decisions, higher levels of satisfaction with problem solving
processes, better time-to-satisfaction ratio and more cohesive multi-organization groups.
Résumé ….....
Ce rapport traite de l’exécution de la Tâche 3 du projet intitulé : « Recherche par la simulation in
vivo sur la prise de décision partagée des méta-organisations », une des composantes du
programme du Fonds pour l’innovation technologique (FIT) relatif à la collaboration métaorganisationnelle, qui a été conçu afin d’améliorer la compréhension des défis auxquels font face
les Forces canadiennes (FC). L’objectif de ce volet particulier est de mener une recherche de base
sur le partage des décisions au moyen d’études de cas, d’exercices et de simulations. La Tâche 3
consistait à élaborer le cadre de partage des décisions in vivo et d’en faire l’essai. À ce stade, la
recherche visait à démontrer que le modèle, lorsqu’il est mis en œuvre in vivo, peut aider à
améliorer les processus de résolution des problèmes et leurs résultats, notamment des décisions de
meilleure qualité, de hauts niveaux de satisfaction en ce qui touche les processus de résolution des
problèmes, un meilleur rapport temps-satisfaction et des groupes multi-organisationnels plus
cohésifs.
'5'&&66&5
i
Executive summary
Research Using In Vivo Simulation of Meta-Organizational
Shared Decision Making (SDM) – Task 3: Testing the Shared
Decision Making Framework in vivo
Louise Lemyre 1 et al.; DRDC CSS CR 2011-32; Defence R&D Canada – CSS.
Introduction: This report documents the completion of Task 3 of the work stream “Research
Using in Vivo Simulation of Meta-Organizational Shared Decision Making (SDM)”, one
component of the Technology Innovation Fund (TIF) program on Meta-organizational
Collaboration that has been designed to assist in understanding challenges faced by the Canadian
Forces (CF). The objective of the stream is to conduct basic research into shared decision making
through the analysis of case studies, exercises and simulations.
The objective of Task 3 is to test the shared decision making framework in vivo.
Method: Headed by Dr. Lemyre, the Gap-Santé research team at the University of Ottawa
created a Model for Inter-organizational Problem Solving under Task 1 of the project. Task 2
involved the development of a research plan for an in vivo simulation experiment as well as
qualitative interviews. Task 3 comprised testing the shared decision-making (SDM) framework
using qualitative interviews with key decision makers having played a role in major events and
testing an in vivo experimental simulation of shared decision making in a complex scenario.
Results: The research at this stage is to demonstrate that the model when implemented in vivo
can produce improvements in problem solving processes and outcomes such as better quality
decisions, higher levels of satisfaction with problem solving processes, better time-to-satisfaction
ratio and more cohesive multi-organization groups.
1
Dr. Lemyre is the McLaughlin Research Chair on Psychosocial Aspects of Risk and Health, Director of
GAP-Santé at the Institute of Population Health, and Professor of Psychology, Faculty of Social Science,
University of Ottawa
ii
'5'&&66&5
Sommaire .....
Recherche sur le partage de décision des méta-organisations en
utilisant la simulation in vivo – Tâche 3 : Essai du cadre de
partage des décisions in vivo
Louise Lemyre 2 et autres ; CSS RDDC CR 2011-32 ; R&D pour la défense
Canada – CSS.
Introduction: Ce rapport traite de l’exécution de la Tâche 3 du projet intitulé : « Recherche par
la simulation in vivo sur la prise de décision partagée des méta-organisations », une des
composantes du programme du Fonds pour l’innovation technologique (FIT) relatif à la
collaboration méta-organisationnelle, qui a été conçu afin d’améliorer la compréhension des défis
auxquels font face les Forces canadiennes (FC). L’objectif de ce volet particulier est de mener
une recherche de base sur le partage des décisions au moyen d’études de cas, d’exercices et
de simulations.
La Tâche 3 a pour objectif de faire l’essai du cadre de partage des décisions in vivo.
Méthode : Sous la direction de Mme Lemyre, Ph. D., l’équipe de recherche GAP-Santé de
l’Université d’Ottawa a créé un modèle de résolution inter-organisationnelle de problèmes dans
le cadre de la Tâche 1 du projet. La Tâche 2 consistait à élaborer un plan de recherche pour une
expérience de simulation in vivo ainsi que des entrevues qualitatives. La Tâche 3, quant à elle,
portait sur l’essai du cadre de partage des décisions au moyen d’entrevues qualitatives réalisées
auprès des principaux décideurs ayant joué un rôle dans des activités d’importance et sur l’essai
d’une simulation expérimentale in vivo du partage des décisions dans un scénario complexe.
Résultats : À ce stade, ;a recherche visait à démontrer que le modèle, lorsqu’il est mis en œuvre
in vivo, peut améliorer les processus de résolution de problèmes et leurs résultats, notamment des
décisions de meilleure qualité, de hauts niveaux de satisfaction en ce qui touche les processus de
résolution des problèmes, un meilleur rapport temps-satisfaction et des groupes multiorganisationnels plus cohésifs.
2
Mme Lemyre, Ph. D., est titulaire de la Chaire de recherche McLaughlin sur le risque psychosocial,
directrice de GAP-Santé, à l’Institut de santé des populations et professeure de psychologie à la faculté des
sciences sociales de l’Université d’Ottawa.
'5'&&66&5
iii
Table of contents
Abstract …….. ................................................................................................................................. i
Résumé …..... ................................................................................................................................... i
Executive summary ......................................................................................................................... ii
Sommaire ....................................................................................................................................... iii
Table of contents ............................................................................................................................ iv
List of figures ............................................................................................................................... viii
List of tables ................................................................................................................................... ix
Acknowledgements ......................................................................................................................... x
Preface ..... xi
1
Introduction............................................................................................................................... 1
1.1
Task 3 objective and components.................................................................................. 1
1.2
Research objectives, questions and strategy.................................................................. 2
1.3
Overview of report sections........................................................................................... 3
2
Component 1 – Qualitative interviews with decision makers................................................... 5
2.1
Overview of research design ......................................................................................... 5
2.2
Ethics approval .............................................................................................................. 5
2.3
Interview guide development ........................................................................................ 5
2.4
Recruitment of interviewees.......................................................................................... 6
2.5
Data collection............................................................................................................... 7
2.6
Data analysis.................................................................................................................. 7
2.7
Next steps ...................................................................................................................... 7
3
Component 2 – In vivo simulation experiment........................................................................ 9
3.1
Overview of research design ......................................................................................... 9
3.1.1
Description of variables ................................................................................ 10
3.2
Ethics Approval ........................................................................................................... 10
3.2.1
Ethics approval process................................................................................. 11
3.2.2
Data storage and participant identifiers......................................................... 11
3.3
Determining scenario complexity................................................................................ 12
3.3.1
Scenario development process ...................................................................... 13
3.3.1.1
Scenario design and selection considerations............................. 13
3.3.2
Scenario assessment process ......................................................................... 14
3.3.2.1
Identification and selection of reviewers .................................... 14
3.3.2.2
Scenario rating assessment questionnaire................................... 15
3.3.3
Scenario assessment results........................................................................... 15
3.3.3.1
Focus group comments and observations on scenarios .............. 16
3.3.3.2
Written comments and observations on scenarios...................... 17
iv
'5'&&66&5
3.4
3.5
3.6
3.7
3.8
3.9
3.10
4
3.3.3.3
Quantitative Analysis of Scenario Ratings ................................. 19
3.3.3.4
Summary of findings .................................................................. 21
3.3.4
Rating scale refinement for future research purposes ................................... 21
Instrument development .............................................................................................. 22
3.4.1
Dependent measure instrument development ............................................... 22
3.4.2
Background questionnaire............................................................................. 22
3.4.3
Consistency of interpretation instrument ...................................................... 22
3.4.4
Decision quality rating instrument ................................................................ 23
Development and assessment of simulation tasks ....................................................... 23
3.5.1
The Shared Decision-Making Model as it relates to simulation tasks .......... 23
3.5.2
Rationale for assessing simulation tasks ....................................................... 24
3.5.3
Task development and refinement process ................................................... 24
Development of multimedia simulation ...................................................................... 25
3.6.1
Design considerations ................................................................................... 25
3.6.2
Creating the simulation ................................................................................. 26
3.6.3
Multimedia components................................................................................ 26
PODS session roles, responsibilities and materials ..................................................... 28
3.7.1
Facilitator role and materials......................................................................... 29
3.7.2
Controller role and materials......................................................................... 30
3.7.3
Observer role and materials .......................................................................... 30
3.7.3.1
Observation forms ...................................................................... 31
3.7.4
Participant materials...................................................................................... 32
3.7.5
Technical and logistical support.................................................................... 33
Recruitment ................................................................................................................. 33
3.8.1
Communication Process................................................................................ 36
Equipment, facilities, software, data collection and storage ....................................... 37
3.9.1
Rationale for use of conference software...................................................... 37
3.9.2
Software selection process ............................................................................ 38
3.9.3
Technical testing of equipment, facilities, and simulation delivery system.. 39
3.9.3.1
Technical testing and equipment and facilities reconfiguration............................................................................... 39
3.9.3.2
Technical testing of simulation system – summary of
solutions...................................................................................... 40
3.9.4
Equipment and facilities requirements and set-up ........................................ 41
3.9.5
NEFSIS set-up and administration................................................................ 42
3.9.6
Data collection and storage ........................................................................... 42
Pilot testing.................................................................................................................. 42
3.10.1 Overview of the pilot testing......................................................................... 42
3.10.2 Pilot testing results ........................................................................................ 44
3.10.2.1 Content and process.................................................................... 44
Next steps: Running the In Vivo simulation........................................................................... 46
'5'&&66&5
v
4.1.1
4.1.2
4.1.3
Scheduled sessions ........................................................................................ 46
Data collection methods and planned analysis.............................................. 46
4.1.2.1
Component 1 – Qualitative interviews with decision makers .... 46
4.1.2.2
Component 2 – In vivo simulation experiment .......................... 47
Modelling communication and decision making functions .......................... 47
References ..... ............................................................................................................................... 48
Annex A .. Qualitative Interview Guide ........................................................................................ 50
Annex B... Interview Invitation Letter........................................................................................... 53
Annex C... Interview Consent Form .............................................................................................. 55
Annex D .. Ethics Approval Letter ................................................................................................ 57
Annex E... Invitation Email for Recruitment................................................................................. 58
Annex F ... Participant Consent Form ............................................................................................ 60
Annex G .. Task Questionnaires .................................................................................................... 63
G.1 Open Pod Task #1 Questionnaire ................................................................................ 63
G.2 Open Pod Task #2 Questionnaire ................................................................................ 70
G.3 Closed Pod Task #1 Questionnaire.............................................................................. 76
G.4 Closed Pod Task #2 Questionnaire.............................................................................. 80
Annex H .. Background Information Questionnaire ...................................................................... 84
Annex I .... Confidentiality / Intellectual Property Agreement ...................................................... 86
Annex J.... Participant ID & Information Linking Form ............................................................... 87
Annex K .. Task Worksheets ......................................................................................................... 88
K.1 Open Pod Collaboration Task Worksheet #1 .............................................................. 88
K.2 Open Pod Collaboration Task Worksheet #2 .............................................................. 93
K.3 Open Pod Coordination Task Worksheet #1 ............................................................... 99
K.4 Open Pod Coordination Task Worksheet #2 ............................................................. 105
K.5 Closed Pod Collaboration Task Worksheet #1.......................................................... 112
K.6 Closed Pod Collaboration Task Worksheet #2.......................................................... 117
K.7 Closed Pod Coordination Task Worksheet #1........................................................... 122
K.8 Closed Pod Coordination Task Worksheet #2........................................................... 126
Annex L... Consistency of Interpretation Instrument .................................................................. 130
Annex M.. Scenario Assessment Guide Contents ....................................................................... 132
M.1 Scenario Assessment Guide: Scenario Complexity Rating Instructions .................. 132
M.2 Scenario Assessment Guide: Confidentiality / Intellectual Property Agreement..... 133
M.3 Scenario Assessment Guide: Background Information Questionnaire..................... 134
M.4 Scenario Assessment Guide: Definitions of Simple, Complicated and Complex .... 135
M.5 Scenario Assessment Guide: Blackout Scenario ...................................................... 136
M.6 Scenario Assessment Guide: Scenario Assessment Questionnaire – Blackout
Scenario ..................................................................................................................... 141
vi
'5'&&66&5
M.7
M.8
M.9
M.10
Scenario Assessment Guide: Mail Scenario ............................................................. 144
Scenario Assessment Guide: Scenario Assessment Questionnaire – Mail Scenario 150
Scenario Assessment Guide: Train Derailment Scenario......................................... 153
Scenario Assessment Guide: Scenario Assessment Questionnaire – Train
Derailment Scenario .................................................................................................. 158
Annex N .. Decision Quality Rating Instrument .......................................................................... 161
Annex O .. PODS Participant Workbook Materials .................................................................... 163
Briefing Information.................................................................................................. 163
Participant Profile Instructions .................................................................................. 163
Gapville Profile ......................................................................................................... 163
Gapville in Depth ...................................................................................................... 165
Situation Report #1.................................................................................................... 168
Public Health Agency Press Release Inject............................................................... 170
Situation Report #2.................................................................................................... 171
Facebook Posts Inject ................................................................................................ 173
Twitter Feed Inject .................................................................................................... 175
Debriefing.................................................................................................................. 176
Annex P ... Inter-GAP In Vivo Session Materials........................................................................ 177
P.1
Briefing Slide Deck ................................................................................................... 177
P.2
Controller Script ........................................................................................................ 178
P.3
Session Administration Information Form ................................................................ 188
P.4
Technical and Process Issues Form ........................................................................... 190
Annex Q .. Recruitment Documents ............................................................................................ 193
Q.1 Student Recruitment Pool Posting............................................................................. 193
Q.2 Participation Invitation Letter.................................................................................... 194
Q.3 Phone Confirmation Script ........................................................................................ 195
Q.4 Reminder E-mail ....................................................................................................... 196
Q.5 Thank You Letter (pilot testing)................................................................................ 199
Q.6 Thank You Letter (senior level participants)............................................................. 201
Annex R... Full Pod Session Equipment Requirements............................................................... 203
List of symbols/abbreviations/acronyms/initialisms ................................................................... 206
'5'&&66&5
vii
List of figures
Figure 1: Model of inter-organizational problem solving .............................................................. 1
Figure 2: Overview of research strategy.......................................................................................... 3
Figure 3: Overall session composition ......................................................................................... 10
Figure 4: Factors and elements contributing to situation complexity........................................... 13
Figure 5: Generic stages of problem-solving ............................................................................... 24
Figure 6: Screenshots from the Inter-GAP In Vivo System scenario injects ............................... 27
Figure 7: Simulated city of Gapville utilized in the Inter-GAP In Vivo System.......................... 28
Figure 8: A typical POD room configuration............................................................................... 41
Figure 9: Review and Pretesting Stages ....................................................................................... 43
viii
'5'&&66&5
List of tables
Table 1: Inclusion criteria for selecting reviewers ....................................................................... 14
Table 2: List of Participant Organizations.................................................................................... 14
Table 3: Mean Level of Complexity by Scenario......................................................................... 19
Table 4: Means and Modes of Factors and Elements by Scenario ............................................... 19
Table 5: Inclusion and Exclusion Criteria for Expert Recruitment .............................................. 33
Table 6: Inclusion and Exclusion Criteria for Pre-testing and Pilot Sessions .............................. 34
Table 7: Summary of Participants Required to Complete the PODS Experiment ....................... 35
Table 8: Inclusion and Exclusion Criteria for Full Sessions ........................................................ 35
Table 9: Overview of the Pilot Tests ............................................................................................ 44
'5'&&66&5
ix
Acknowledgements
As principal investigator, Dr. Louise Lemyre wishes to acknowledge the funding made available
by Defence Research and Development Canada (DRDC) for this important research initiative. We
also want to thank all of our various partners and collaborators, especially those of DRDC,
Canadian Forces and emergency response of all sectors who have shared their expertise in the
development and refinement of the various components of the in vivo simulation experiment.
Special thanks are also offered to those individuals and organizations that have assisted in
recruiting participants for the experiment and to the participants themselves who have given their
time to attend.
x
'5'&&66&5
Preface
This document represents the initial draft of Task 3: Testing the Shared Decision Making
Framework in vivo, and is submitted for review before a final document is prepared using the
DRDC supplied template for Contractor reports. The work has been completed for Defence
Research and Development Canada (DRDC) as part of the contract deliverable defined in the
project entitled Research Using In-Vivo Simulation of Meta-Organizational Shared Decision
Making (SDM), Contract No.: W7714-083659/001/SV.
The document is a technical report that provides detailed descriptions of all of the materials,
instruments, procedures and processes created to implement both the qualitative interview
component of the study and the in vivo simulation exercise. The results of the scenario assessment
process for determining complexity are included. Also included in this report are results from the
multiple pilot testing of the simulation using different configurations of participants and pods and
the plans formulated for conducting simulation exercise sessions with senior decision makers in
early April and May 2011. A follow-on report is planned and will include detailed analyses and
results for both the qualitative interview component and the senior level sessions of the in vivo
experiment.
Document Distribution and Confidentiality
Document distribution and confidentiality protocols as specified in the contract noted above will
apply to this document. Please contact Dr. Louise Lemyre, Principal Investigator, University of
Ottawa, at louise.lemyre@uOttawa.ca should a change in protocols be requested. Please quote
with due reference to Lemyre et al. 2011, Report on Research Using in Vivo Simulation of MetaOrganizational Shared Decision Making (SDM) – Task 3: Testing the Shared Decision Making
Framework in vivo.
'5'&&66&5
xi
This page intentionally left blank.
xii
'5'&&66&5
1
Introduction
This draft report documents the completion status and results to-date of Task 3: Testing the
shared decision making framework in vivo. Task 3 is one of five tasks to be undertaken to
conduct basic research into shared decision making through the analysis of case studies, exercises
and simulations. It builds on work completed under Task 1: Synthesis of Case Studies to form a
SDM Framework (see figure below) and reflects the outputs of Task 2: Development of an
experimental plan for in vivo exercise and simulation. Refer to report entitled, Research Using In
Vivo Simulation of Meta-organizational Shared Decision Making (SDM) Task 1: Synthesis of
Case Studies to form a SDM Framework, (Lemyre et al., 2009) for details on Task 1.
Figure 1: Model of inter-organizational problem solving
This report focuses on the implementation procedures and protocols for both the qualitative
interview component of Task 3 and the in vivo simulation experiment. Detailed descriptions are
provided of all materials utilised during the course of the testing undertaken to-date and the
interview processes. While all materials and processes have been validated through extensive
rounds of pilot testing, experimental sessions with senior decision-makers are presently being
planned for April and May 2011. Given the required timing of this report, results for the
experimental component will be limited in this report to relevant themes and observations from
the multiple pilot tests that have been undertaken during Task 3. All of the qualitative interviews
have been conducted but these are in the transcription and coding processes. After completion of
these activities, analyses will be undertaken and findings integrated with those from the in vivo
experimental sessions with senior officials.
1.1
Task 3 objective and components
The objective of Task 3 is to test the shared decision making framework in vivo. From the results
of Task 1, identify scenarios that challenge the SDM framework, as well as the ICS framework, if
the latter differs from the SDM framework. These scenarios will be exercised within the
simulated environment designed in Task 2. Participants in the simulations will include individuals
'5'&&66&5
1
from multiple levels of government, multiple jurisdictions and multiple disciplines according to
the requirements of the particular scenario being exercised.
1.2
Research objectives, questions and strategy
The research objectives, questions and strategy were developed as part of Task 2 and a full
discussion can be found in the report entitled Research Using In Vivo Simulation of Metaorganizational Shared Decision Making (SDM) Task 2: Development of an experimental plan for
in vivo exercise and simulation (Lemyre et al., 2010). The figure below summarizes this
information and is re-presented for reader convenience.
The research at this stage is to demonstrate that the model when implemented in vivo can produce
improvements in problem solving processes and outcomes such as better quality decisions, higher
levels of satisfaction with problem solving processes, better time-to-satisfaction ratio and more
cohesive multi-organization groups.
2
'5'&&66&5
Model for Inter-organizational Problem Solving
Research Objective:
7REHWWHUXQGHUVWDQGKRZSUREOHPVROYLQJSURFHVVHVDQG
RXWFRPHVGXULQJFRPSOH[H[WUHPHHYHQWVFDQEHLPSDFWHGE\
WKHDSSURDFKHVXVHGWRSUREOHPVROYHDQGWKHVWUXFWXUHRIWKH
PXOWLRUJDQL]DWLRQDOHQYLURQPHQW
Research Questions:
Q1::KDWHIIHFWGRHVWKHDSSURDFKWRPXOWLRUJDQL]DWLRQDOSUREOHP
VROYLQJKDYHRQSUREOHPVROYLQJSURFHVVHVDQG RXWFRPHV"
Q2: :KDWHIIHFWGRHVWKHPXOWLRUJDQL]DWLRQDOHQYLURQPHQWKDYHRQ
SUREOHPVROYLQJSURFHVVHVDQGRXWFRPHV"
Component 1:
•
4XDOLWDWLYHLQWHUYLHZVZLWKNH\GHFLVLRQ
PDNHUV
•
$SSUR[LPDWHO\LQWHUYLHZV
•
,GHQWLI\NH\IHDWXUHVFRJQLWLYHVWUXFWXUHV
DQGSDWWHUQVXVHGWRJXLGHGHFLVLRQVDQG
DFWLRQVDERXWH[WUHPHHYHQWV
Component 2:
•
,QYLYRVLPXODWLRQZLWKFRPSOH[VFHQDULR
•
VHVVLRQVRIWKUHHSRGV
•
&RQWUROYDULDEOHVLWXDWLRQFRPSOH[LW\
•
,QGHSHQGHQWYDULDEOHVSUREOHPVROYLQJ
DSSURDFKPXOWLRUJDQL]DWLRQDOHQYLURQPHQW
•
'HSHQGHQWYDULDEOHVSUREOHPVROYLQJ
SURFHVVHVDQGRXWFRPHV
Analysis and Integration of Findings
Figure 2: Overview of research strategy
1.3
Overview of report sections
This document is organized into three main sections with various sub-sections. The content for
each of these sections is briefly described below.
SECTION 1
Introduction
• This section outlines the objectives of Task 3 and its qualitative and experimental
components.
• Details are provided concerning the study`s research objectives, questions and strategies.
'5'&&66&5
3
• The three sections of the report and what each section will contain are described.
SECTION 2
Component 1 – Qualitative interviews with decision makers
• This section outlines the qualitative interview research design and its rationale.
• Details are provided concerning the ethics approval process, the interview guide
development and its contents, and the interviewee recruitment procedure.
• The data collection and analysis process is described.
• Next steps for the qualitative interviews focus on the directions of the final analyses.
SECTION 3
Component 2 – In Vivo Simulation experiment
• This section outlines the in vivo simulation experimental research design, and briefly
describes the control, independent, and dependent variables.
• Details are provided concerning ethics approval, the ethics approval process as well as
proper data storage and maintaining participant confidentiality through the use of participant
identifiers.
• The rational for rating scenario complexity and its relation to the shared decision making
model are discussed. The scenario development process, the scenario assessment and its
results, as well as the scenario selection and refinement process are presented. Refinement
of the rating scale for future research purposes is discussed.
• Developed instruments such as dependent measures, the background questionnaire, and the
decision quality rating instrument are presented and described in terms of their development
process, design considerations, review process, and their refinement.
• Development and assessment of simulation tasks are discussed. Details are presented on the
rational for assessing simulation tasks, simulation tasks relation to the shared decision
making model, and the task development and assessment process. Worksheets that
correspond to the chosen tasks are presented.
• Details are provided concerning the development of the multimedia simulations, POD
session roles, responsibilities and materials, recruitment methods, necessary equipment,
facilities and software, and data collection and storage procedures.
• A pilot testing overview is presented along with results and information regarding changes
made to simulation materials.
SECTION 4
Next Steps
• This section provides an overview of the remaining scheduled sessions with senior
emergency management professionals.
• Data collection methods and planned analysis are also described with respect to both the
qualitative interview component and the in vivo simulation component of the project.
• Finally, the upcoming task of modelling communication and decision making functions is
described.
4
'5'&&66&5
2
Component 1 – Qualitative interviews with decision
makers
This section of the report presents an overview of the qualitative interview component of the
project. The following items are described here: overview of the research design, ethics approval,
interview guide development, recruitment, data collection, data analysis, and next steps.
2.1
Overview of research design
Qualitative research methods are often complementary to using quantitative methods. Qualitative
methods elicit explanations, discover patterns, and understand relationships in the data that may
be overlooked when only using quantitative methods. Qualitative methods help go beyond the
scope of quantitative methods (Crotty, 1998; Smith, 2008). The present study uses qualitative
findings from interviews with key stakeholders as a method to identify critical insights into
elements of decision making processes during extreme events in real-life settings retrospectively
(Nja & Rake, 2009) to aid in the planning, validation, analysis and interpretation of the
quantitative in vivo experimental data. The interviews in the present study focus on the general
process of problem solving, during which decision making is considered to be one of the key
stages.
The present study uses the Critical Decision Method approach to interviewing (Klein,
Calderwood & MacGregor, 1989) which has been simplified and adapted by other authors (e.g,
Smith and Dowell, 2000; Nja & Rake, 2009). A semi-structured interview guide consisting of
core questions and probes (Lindlof & Taylor, 2002) was used to conduct the interviews. The
interviewer guided interviewees through the various stages of problem solving that had occurred
during a previous extreme event, drawing out the interviewees’ past experiences and
interpretations. The main strengths and challenges to using this approach were discussed in the
previous report “Research Using In Vivo Simulation of Meta-Organizational Shared Decision
Making (SDM): Task 2: Development of an experimental plan for in vivo exercise and
simulation”.
2.2
Ethics approval
The qualitative interview component of the study obtained ethics approval from the Social
Sciences and Humanities Research Ethics Board of the University of Ottawa. For details on the
application and ethics submission please refer to Section 3.2.
2.3
Interview guide development
A semi-structured interview guide allows new questions to be asked based on the interviewees’
conversational direction (Lindlof & Taylor, 2002). For the present study, a guide was developed
to instruct the administration and implementation of the qualitative interviews. The guide was
used for all interviews to ensure some consistency across interviews and increase the reliability of
findings. The draft guide went through various iterations and pre-testing to ensure that the
'5'&&66&5
5
interview was of an appropriate length (approximately 60 minutes), and that questions collected
appropriate information from interviewees, were easy to understand, and flowed in a logical
fashion.
The questions in the interview guide were based on the current study`s two main research
questions described in Section 1.2. Nine interview questions along with additional probes were
developed to address the two main research questions (see Annex A). Exact wording of the
questions varied during interviews to accommodate the different situations or events being
studied. The interview began with a brief introduction to convey some context to the
interviewees, and was followed by some questions designed to collect some basic, factual
information concerning the interviewees’ background, an overview description of the selected
event, and the selection of one or two challenging situations that had complex and multiorganizational characteristics. Once one or two situations had been selected, the interview
focused on the chosen situations with questions designed to have the interviewee describe and
reflect upon problem identification and definition, solution generation, decision making and
implementation, sharing resources and flexibility, expectations and alternatives, and the three
concepts of coordination, cooperation and collaboration.
2.4
Recruitment of interviewees
The inclusion criteria for participants for the qualitative interviews were that they were seniorlevel decision makers and managers who had participated in planning for and/or managing a
major, extreme event in Canada within the past ten years. They would need to have played a
strategic decision-making role during at least one major event such as the 2010 G-20 Toronto
Summit, 2010 Vancouver Olympic Games, the 2009 H1N1 planning, the 2007 G8 meeting in
Ottawa, or the, SARS outbreak in 2003. Other considerations included level of experience
managing extreme events, and possessing the authority to make decisions regarding allocation of
resources on behalf of their organization with respect to major events. Some potential
interviewees were identified via the professional networks of research team members. Others
were identified through referrals from key senior managers and decision-makers who participated
in the same events.
One goal of the interviews was to ensure that there was some diversity among interviewees with
respect to the type of decision/command structures that were characteristic of their organizations.
As a result, an effort was made to recruit key decision makers from a variety of organizations that
follow different command structures. Three types of organizations were targeted:1) military
organizations that employ a C2 structure (e.g. Canadian Forces); 2) service organizations that rely
primarily on the Incident Command System (ICS) approach to planning for and responding to
extreme events (e.g., EMO); and 3) organizations that follow less hierarchical, more distributed
and less prescriptive decision-making processes (e.g., NGOs, community organizations, health
authorities).
Given the inherent challenges involved in the recruitment of senior-level decision makers, a
flexible recruitment process was developed involving various stages and procedures which
included identifying potential interviewees, making initial contact (see Annex B for the e-mail
invitation), securing agreement to participate, scheduling interviews, obtaining informed consent,
and conducting the interviews.
6
'5'&&66&5
2.5
Data collection
All interviews were conducted by senior-level interviewers with advanced knowledge and
experience with qualitative interviewing techniques. Prior to the interview starting, interviewees
were provided with detailed information about the study via an information sheet that
accompanied the consent form. The information and consent form was approved by the
University of Ottawa Research Ethics Board and is in compliance with the Tri-Council Policy
Statement: Ethical Conduct for Research Involving Humans (TCPS). The interviewer would
guide the interviewee through the informed consent process, asking them if they had any
questions or required any further clarification prior to signing the form indicating their consent to
participate in the interview. (See Annex C for the participant consent form).
Once informed consent was obtained, the interviewer proceeded with the interview using the
interview guide. (See Annex A for the qualitative interview guide). Interviews were semistructured, with open-ended questions asked, followed by probes designed to elicit more detailed
information. All interviews were audio-taped, with tapes later transcribed verbatim.
As of the timing of this report, ten participants had been recruited and interviewed, eight in the
Toronto area and two in the National Capital Region, from military, ICS and non-ICS
organizations. Interviews lasted on average 60 minutes in duration.
2.6
Data analysis
In qualitative research, often a larger emphasis is placed on the depth and details of the data than
on the breadth and representativeness of data. Each qualitative interview is treated as a rich
source of information that broadens the researcher’s understanding, that when combined with the
information collected from other interviews will demonstrate patterns and themes, along with
explanations and examples.
The verbatim transcripts of the interviews were imported into an analytic software program
designed to assist with categorizing, and assembling qualitative information so that trends and
patterns can be analyzed. Initially, analysis of the transcripts began with identifying meaningful
units of information that are related to situation complexity (i.e., simple, complicated, and
complex). This was followed by identifying units of information by expected themes (e.g., multi
organizational environment, approach to problem solving, problem solving stage) and emerging
themes. Data coding (i.e., assigning text or units of information to categories) was conducted in a
cascading manner, where data was categorized into a small, broad number of categories.
2.7
Next steps
At the timing of this report, recorded interviews are currently being transcribed. Once the
transcriptions have been completed, the detailed analyses will begin. All transcripts will be
imported into analytic software and follow the analysis process as described in Section 2.6 Data
Analysis. Once a small number of broad categories are identified from the interviews, these
categories will be coded into additional levels of subcategories to specify certain themes. To
ensure reliability of the coding scheme and analysis, two researchers will recode 10% of the
interviews, and then these coding results will be compared. Finally, results will be linked with the
'5'&&66&5
7
quantitative experimental in vivo component, and will aid in the interpretation of the quantitative
findings.
8
'5'&&66&5
3
Component 2 – In vivo simulation experiment
This section describes the second major component of the project – the in vivo simulation
experiment. An overview of the research design for the in vivo experiment is presented along
with a description of the ethics approval process. This portion of the report also describes our
methods for determining situational complexity; the instruments used in the experiment; scenario
and task development; the development of the final multimedia version of the simulation
scenario; the roles, responsibilities and materials associated with each of the actors involved in
the experiment; the recruitment process; the equipment, facilities, software and set-up required to
run the experiment; as well as a description of the pilot testing phases of the experiment.
3.1
Overview of research design
In order to test the model for inter-organizational problem solving, an in vivo simulation
experiment was developed. This simulation experiment is in keeping with Jonathon Crego’s work
on the HYDRA simulation training system (Alison & Crego, 2008); however, the Inter-GAP In
Vivo System developed by Lemyre et al. (2010) looks not only at intra-organizational problem
solving but also at inter-organizational problem solving. The research design assumes a complex
situation within which decision makers have to undertake a number of tasks characterized as
either “coordination” or “collaboration”.
During an Inter-GAP simulation system session, groups of three participants are organized into
pods which are equipped with communication equipment. These teams work through a simulated
emergency event while their communications, interactions (both within pods and between pods)
as well as the decision making processes are recorded for later analysis. Three of the teams are
able to communicate between the pods, while a fourth team is used as the control group, and is
not given the communication tools to interact with the other pods (see Figure 3 below for the
overall session composition). The session is delivered via a separate control room using video
conferencing software, where the scenario, tasks, and injects are delivered according to a set
schedule. Facilitator-led briefing and debriefing sessions are used to orient the participants to the
materials and technology used in the experiment and to gain informed consent from each
participant.
'5'&&66&5
9
Control
Room
2EVHUYDWLRQRQO\
2EVHUYDWLRQ,QIR
POD 1
POD 2
1RLQWHUSRG
FRPPXQLFDWLRQ
POD 3
POD 4
,QWHUSRGFRPPXQLFDWLRQ
Figure 3: Overall session composition
3.1.1
Description of variables
The independent variables that were manipulated during the experiment are a) Approach to
problem solving, and b) Multi-organizational environment. Approach to problem solving includes
two levels (Coordinating; Collaborating). Multi-organization environment includes two levels of
pod composition (Homogeneous organizations; Mixed types of organizations), and two levels of
inter-pod interaction (Closed interaction; Open interaction).
The main dependent variables are problem solving processes and outcomes such as decision
quality, satisfaction with problem solving process according to actors and to external panel, task
cohesion, participation, time and agreement. For a more detailed description of the independent
and dependent variables see the previous report “Research Using In Vivo Simulation of MetaOrganizational Shared Decision Making (SDM): Task 2: Development of an experimental plan
for in vivo exercise and simulation”.
3.2
Ethics Approval
Before the execution or testing of any of the components of the in vivo simulation of metaorganizational shared decision making project, ethics approval had to be granted. Therefore,
observing the guidelines and regulations from the University of Ottawa Ethics Research Board
and the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS),
the ethics application involved a meticulous and arduous effort from the project team to submit a
complete application under the minimal risk review process category for the experimental part,
and as a modification for a research project for the qualitative component.
As a result, the University of Ottawa Research Ethics Board complying with the Tri-Council
Policy Statement: Ethical Conduct for Research Involving Humans (TCPS) and other legislations,
examined and approved the application for ethical approval of the Risk Management and
10
'5'&&66&5
Governance: Understanding Problem Solving and Decision Making (File # 08-10-31), granted as
of September 14th 2010 (See Annex D for the Ethics Approval Letter).
3.2.1
Ethics approval process
The application for ethics approval was submitted as a minimal risk review, which is used for
studies that pose minimal risk to the participants rights and safety, and that comply with the
principles of free and informed consent, privacy and confidentiality, appropriate balance of risks
and benefits, between others. The ethics application addressed research protocols, methodologies
and processes, recruitment and selection of participant’s processes, benefits and risks posed to the
participants, the consent process, as well as the privacy, confidentiality and anonymity of the
participants.
Consequently, the ethics application included the submission of the following documents (Annex
E, & F), for both the qualitative and quantitative components:
• An invitation email for recruitment
• A participant consent form
Specifically the qualitative component required the submission of the qualitative interview guide
(See Annex A), while the quantitative in vivo simulation experiment required the participants’
questionnaires (See Annex G), and the description of the scenario injects (Please refer to Section
3.6).
3.2.2
Data storage and participant identifiers
In compliance with the Tri-Council Policy Statement: Ethics Conduct for Research Involving
Humans (2008, p.44), “The duty of confidentiality includes obligations to protect the data from
unauthorized access, use, disclosure, modification, loss or theft”. Different procedures were
developed to ensure that all data, including written records, consent forms, questionnaires, video
and audio tapes will be maintained in a locked filing cabinet within a locked office within the
secure zone of the Faculty of Social Sciences research centre at DMS 3rd floor. All electronic
files (including digital audio and video recordings) with participant data are being encrypted and
password protected. The files will be stored on a dedicated server to which only the research team
has access. The data will be conserved for a period of 10 years once the relevant publications
have been produced. During the conservation period, all hard copy data (e.g., questionnaires) will
be stored in a locked filing cabinet at the University of Ottawa. Electronic files will remain
encrypted, password protected and stored on a server to which only the research team has access.
Once the conservation period is over, the hard copy data will be shredded and the electronic files
will be deleted via secure deletion methods.
Each participant will be labelled with a different participant identifier to ensure participant
confidentiality and proper tracking of data. Participant identifiers consist of five numbers: the first
two numbers indicate the session number, the third number indicates the session type (1coordination, 2- collaboration), the fourth number indicates the pod number, and the fifth and last
number indicates the participant number (up to three participants per pod). For example, the
'5'&&66&5
11
number “05112” indicates this participant is part of session five, is in a coordination session, is
part of pod one and is participant two in this pod. These identifiers will be used to label each
participant’s documents. These documents include: participant consent forms (see Annex F),
background questionnaires (see Annex H), confidentiality and intellectual property agreements
(see Annex I), participant identification and information linking forms (see Annex J), task
worksheets (see Annex K), task questionnaires (see Annex G), and consistency of interpretation
questionnaires (see Annex L).
These documents will be stored in a secure environment indicated in Section 3.9.6 Data
Collection and Storage. Data that contains participant’s names, such as consent forms,
background questionnaires, confidentiality and intellectual property agreements, and participant
identification and information linking forms, will be stored in a different location than the
participants’ worksheets and questionnaires to avoid linking data with individual participants.
3.3
Determining scenario complexity
The PODS experiment is designed to measure the impact that the approach to multiorganizational decision-making and the types of multi-organizational environments have on
problem solving processes and outcomes during complex extreme events. Thus, the experimental
design involved holding situational complexity constant as a control variable, while the following
independent variables were manipulated: Approach to problem solving (collaboration or
coordination), as well as Multi-organizational environments (homogeneous or mixed; and open or
closed).
In keeping with the experimental design, the simulation needed to reflect a high level of
complexity characteristic of an extreme event. This was achieved by keeping the factors and
elements that contribute to situation complexity at the forefront of the writing process. These
factors and elements are a part of the Model for Inter-Organizational Problem Solving devised
during Task 1 of this project (see Figure 4 below for a breakdown of these factors and elements
within the model). Rating scales were subsequently developed and utilized by expert reviewers to
ensure that a complex event was indeed being portrayed in the simulation.
12
'5'&&66&5
Figure 4: Factors and elements contributing to situation complexity
3.3.1
Scenario development process
Three separate scenarios (see Annex M) were developed for the scenario assessment process. A
detailed simulated city was used to situate these scenarios. This fictional city, entitled Gapville, is
a mid-sized Canadian city, located close to an international border, and features all of the
requisite infrastructure, populations, and development to provide a simulation with a high degree
of realism. This intricate backdrop became the backbone for three simulation storylines: 1) a train
derailment and chemical contamination scenario; 2) a cyber attack and blackout scenario; and, 3)
a radiological ‘dirty bomb’ scenario. All three scenarios were written using peer-reviewed
research, salient technical reports, and existing federal, provincial, and local emergency plans to
guide the accuracy of the events in the scenarios.
3.3.1.1
Scenario design and selection considerations
Three main design considerations guided the development of the scenarios – (1) degree of
complexity, (2) accuracy, and (3) plausibility were paramount considerations not only in
constructing the scenarios, but also in the assessment and selection process. The importance of
designing a scenario with a high degree of complexity has already been noted, but accuracy and
plausibility were also essential design considerations. Accuracy was wanted (especially in
technical or scientific areas) since it adds to the realism of the scenario and encourages
'5'&&66&5
13
participants to engage with the content and not “fight” it. Disagreements among participants
about technical details may cause them to lose focus on the tasks to be completed. Also, given the
in vivo aspect of the study, the scenario selected must have the potential to actually occur.
Scenario plausibility was needed for participants to willingly suspend their disbelief during the
experiment.
3.3.2
Scenario assessment process
For this experiment, three scenarios were developed and assessed using focus groups and
interviews held with experts in the field of emergency management. The scenarios were validated
for accuracy, plausibility and degree of complexity using a scenario rating assessment
questionnaire. The scenarios assessed included (1) a train derailment and subsequent chemical
contamination scenario; (2) a cyber attack scenario occurring in conjunction with a blackout and
extreme winter weather; and (3) a radiological attack, or ‘dirty bomb’ scenario.
3.3.2.1
Identification and selection of reviewers
Raters of scenarios were screened to meet a set of inclusion criteria (see Table 1). Reviewers were
chosen based on their expert level knowledge of emergency management and their membership in
organizations that respond to or manage extreme events. More than fifteen invitations were sent
to potential participants. Eight experts (seven males and one female) participated in the expert
rating session held at the University of Ottawa on October 17th, 2010 (see Table 2 for a list of
participant organizations). Additional meetings were held with participants on an individual basis
as schedules allowed. Participating organizations included Federal, Provincial and Municipal
agencies responsible for health, safety, security and defence tasks. Data from the rating
questionnaires were entered into SPSS and frequencies were analyzed.
Table 1: Inclusion criteria for selecting reviewers
Inclusion Criteria
([SHUWNQRZOHGJHRIHPHUJHQF\PDQDJHPHQWDQGUHVSRQVHWRH[WUHPHHYHQWV
0HPEHURIDQRUJDQL]DWLRQLQYROYHGZLWKHPHUJHQF\UHVSRQVHRUPDQDJHPHQW
$EOHDQGZLOOLQJWRYROXQWHHUWRSDUWLFLSDWHLQWKHVWXG\
4. $EOHDQGZLOOLQJWRSDUWLFLSDWHLQ(QJOLVK
Table 2: List of Participant Organizations
Sector/Type of Organization
2WKHU)HGHUDO*RYHUQPHQW
'HSDUWPHQWVDQG$JHQFLHV
14
Organizations
• 3XEOLF6DIHW\
• &DQDGLDQ)RUFHV
'5'&&66&5
3URYLQFLDO*RYHUQPHQW
'HSDUWPHQWVDQG$JHQFLHV
• 2IILFHRI'LVDVWHU0DQDJHPHQW0DQLWRED+HDOWK
0XQLFLSDO*RYHUQPHQW
'HSDUWPHQWVDQG$JHQFLHV
• 2WWDZD3XEOLF+HDOWK
• 2WWDZD)LUH'HSDUWPHQW
• &LW\RI2WWDZD
3.3.2.2
Scenario rating assessment questionnaire
A Scenario Assessment Guide (see Annex M) was developed to guide raters through the scenario
assessment process. The reviewers were asked to complete a consent form and a confidentiality
form at the outset of the rating session. Next, raters were given some background information on
the project and given instructions on how to fill out the rating instrument. Prior to beginning the
assessment of the scenarios, these raters also filled out a Background Questionnaire (see Annex
M.3) which contained questions of a socio-demographic nature. The expert reviewers then read
over the three written scenarios and filled out the Scenario Assessment Questionnaire (see Annex
M.6, M.8 & M.10).
The Scenario Assessment Questionnaire was comprised of 22 questions designed to capture a
reader’s understanding of what contributes to the complexity of a given scenario. All three
scenarios were assessed using the same questionnaire. First, reviewers were asked to rate each
scenario as ‘simple’, ‘complicated’ or ‘complex.’ Potential contributions to complexity for a
scenario were described in terms of impact, uncertainty and vulnerability, with specific
challenges or areas noted for each. After reading the scenario, the reviewers were then asked to
rate the degree to which each challenge was evident in the scenario. Answers were given based on
a 5-point Likert scale ranging from 0 (not at all evident) to 4 (strongly evident) for each potential
contribution. Then reviewers were asked which of the factors contributed most to the level of
complexity of a given scenario and to explain their choice of answer. Here, reviewers chose
between ‘impact’, ‘uncertainty’, ‘vulnerability’ or ‘all of the above.’ Reviewers were also asked
to rate the overall level of complexity of the scenario on a Likert scale of 0 (not at all complex) to
4 (very complex). Finally, reviewers rated their level of agreement with the plausibility of the
scenario on a scale of 0 (strongly disagree) to 4 (strongly agree). The results were compared with
data collected from other reviewers to select a suitably complex scenario for use in the simulation
exercise.
Focus groups and meetings with individual reviewers also involved unstructured discussion of the
scenarios. This generated valuable feedback. These discussions allowed experts to address
specific issues of technical and scientific accuracy within the scenarios, and to describe positive
and negative aspects choosing one scenario over another based upon their own experience in
responding to extreme events.
3.3.3
Scenario assessment results
The scenario assessment results include focus group comments, written commentary from the
Scenario Assessment Guide, and the results from the scenario assessment questionnaire.
'5'&&66&5
15
3.3.3.1
Focus group comments and observations on scenarios
At the end of the focus group session, reviewers indicated that the most complex scenario was the
radiological/nuclear Mail Scenario, followed by the Train Scenario, and lastly the ice storm and
cyber attack Blackout Scenario. In addition, reviewers stated that the final scenario should be one
in which many actors can have credible roles.
Focus Group Comments on the Blackout scenario
Reviewers felt that the cyber attack and the ice storm were completely separate streams of activity
in terms of response. One reviewer thought this scenario was reasonably complex, while another
felt we had underplayed the impact on critical infrastructure. Another reviewer thought this
scenario was not realistic enough and stressed that the military would be of last resort only.
Focus Group Comments on the Mail Scenario
Overall, raters felt that the scenario was very complex. Reviewers cited a number of factors
contributing to the high level of complexity in the scenario. These included:
• The closure of the hospital, which would be a major event causing many ripples province
wide
• The issue of public trust
• The emergence of “for-profit experts” (e.g. sale of Prussian blue online)
• Containment issues from the blown out window from the initial explosion
• Cross-jurisdictional issues
• The arrival of international media
• Public affairs and communication issues
• Response would be 125% reactive and still not get it right and, as a result, the intelligence
community will be accused of failure.
Suggestions were made to increase the level of complexity further, to provide an even greater
opportunity for the military to play a significant role. A suggestion was made by one reviewer to
make the decontamination side more evident, stating that, “it really becomes a logistics and
communications nightmare to be able to go to peoples' homes and places of work to collect
contaminated clothing.” It was also suggested that the second event in this scenario would not
have to occur to add to the complexity of the situation. A threat would also draw resources away,
necessitating more help from other levels and organizations.
Raters discussed the potential actors in the scenario. These comments include:
• Once the municipality cedes control, it assumes a support role, posing communication
problems
• The Province would have to make an official request to the Feds for help
• The military is linked in as soon as the Province communicate with the Feds
16
'5'&&66&5
• NGOs would not want to participate, given the nature of the event
• As support to the R.C.M.P., a national security response team would be called in
The timeline of the scenario was questioned by reviewers. Raters stated that first responders (fire)
would know quickly that it was a radiological event as they wear detection gear when on the
scene. Additionally, it was felt that the timeline for this scenario could be reduced to several
hours rather than several days. Reviewers stated that, depending on the type of event, troops
would be available within 6, 12, or 24 hours.
Focus Group Comments on the Train Scenario
One reviewer considered this scenario to be complicated, but not complex because it did not
include a criminal element and because authorities have dealt with these types of situations before
(though perhaps not on the same scale). Reviewers found this scenario to be believable. However,
the reviewers requested more details on demographics, critical infrastructures, existing response
capacity, and size of the community affected. Overall, the two biggest issues reviewers saw with
this scenario were public health and the clean-up (i.e., consequence management).
The timing of the scenario was also questioned. Reviewers stated that responders (i.e. fire) would
discover quite quickly (under an hour) what the chemical was and set up a perimeter as
prevention. Furthermore, reviewers stated that the Hazmat team would arrive on scene in under
an hour as well. Reviewers from Public Safety focused on national and international resources.
They felt that the local community would declare an emergency very quickly and request
assistance and the military would provide security. Reviewers envisioned that the Province would
take the lead at first, followed by Public Safety at the Federal level along with Health Canada.
A suggestion from a reviewer was to make this scenario more complex at the site level by adding
cars containing chemicals that would react with the breached car. Another suggestion for
increasing situational complexity was to add a national security issue (i.e. a terrorist cause).
One of the individual meetings yielded an important observation about the Train Scenario,
namely that while the content of the scenario may be accurate, it is not plausible within a
Canadian context. The problem of plausibility the reviewer spoke of relates to the fact that the
chemical being transported in the scenario is more frequently manufactured and used on site.
Transportation of this chemical is generally avoided, precisely because of the impacts described
in the written scenario. Thus, it was felt that the radiological/nuclear scenario was more plausible.
3.3.3.2
Written comments and observations on scenarios
A number of the reviewers chose to write comments on the scenarios in the margins of their
Assessment Guides. These comments are summarized below.
Written Comments on the Blackout scenario
Reviewers indicated that the timing of interventions throughout the day was more delayed than
they would be in reality. For instance, one reviewer stated that hospital overflow would be
mitigated at an earlier time than indicated in the scenario. A second reviewer pointed out that
'5'&&66&5
17
action on the part of the local community networks would take place earlier than indicated. A
third stated that it took too long to realize that the computers did not work. This reviewer also
expected the power restoration to be outlined as a priority earlier in the scenario. In addition, one
reviewer thought that it was not likely for the community center’s generator fuel to last only one
night and that it would be more likely that patients would be moved out of affected areas and/or
discharged into family care.
Reviewers requested the addition of more specific dates, the geographical size of Gapville, and
the population size of Gapville. Clarification was also requested with respect to the actors
involved, and suggestions were provided by reviewers. For example, one reviewer stated that the
province would provide aid before asking for resources at the federal level. Another reviewer
viewed the armed forces as a last resort resource that was not necessary in this scenario. Another
reviewer disagreed that police would still be at the hospital seven days after the initial crisis.
Finally, one reviewer suggested multiple clarifications on the figures responsible at each stage of
the scenario. For example, the reviewer would prefer the phrase “the head of the hospital” to be
replaced with terms such as “the hospital administrator” or the “executive director.”
Written Comments on the Mail Scenario
Reviewers indicated issues with respect to timing and actors involved in the scenario. More detail
in terms of numbers, scope, capacity and demographics were also requested.
As was the case with the Blackout Scenario, there was a request for clarification in terms of the
responders present. For example, when the ‘dirty bomb’ is sent to the television station, two
reviewers indicated that the police should have been contacting the R.C.M.P. and not the military.
One reviewer suggested that it should be the National CBRN Response Team that should be
intervening rather than the HazMat team and the federal investigators. Furthermore, this reviewer
remarked that the military would not be capable of setting up decontamination sites, as is done in
the scenario. It was suggested that volunteer organizations be used to help handle the flow of
traffic, enforce the evacuation zone, and provide shelter and support for displaced persons. In
terms of identifying the perpetrators responsible for the dirty bomb, one reviewer suggested that it
should be the Feds investigating rather than the local police.
The timeline was also mentioned by reviewers. One reviewer felt that exposure estimates should
have been given much earlier in the scenario. Another reviewer pointed out that it would be
helpful to know the month and year the scenario occurred in. In addition, this reviewer felt that
the timeline was too long and unrealistic. Another reviewer wanted more clarification regarding
the amount of time it took for the screening and treatment process at the hospital. Two reviewers
stated that public notification should occur and a state of emergency should be declared during
the scenario. Finally, a reviewer noted that the military would arrive within 4 hours, not 72.
Written Comments on the Train Scenario
Reviewer comments indicated that they would like more specifics about the scenario. For
instance, specifics were requested regarding the type of respirators paramedics wore (two
reviewers), the size of Gapville, and the number of citizens in Gapville (three reviewers), the
number of people living in the 11km radius of the evacuation zone (one reviewer), and the
weather conditions (one reviewer). Finally, clarification was suggested when using words like
“authorities” to indicate who exactly these authorities were (one reviewer).
18
'5'&&66&5
As in the other two scenarios, there were comments regarding delays in the timeline and the
responders involved. One reviewer suggested that international assistance should be requested,
and three reviewers pointed out that the involvement by the Feds should occur right away rather
than the day after the initial crisis. One reviewer pointed out that a state of emergency should be
declared.
3.3.3.3
Quantitative Analysis of Scenario Ratings
This section describes the quantitative results from the Scenario Assessment Questionnaire. In
terms of the overall level of complexity, it was found that the Train Scenario had the highest
complexity rating. The Mail Scenario was the second most complex and the Blackout Scenario
was the least complex of the three. See the table below for the mean level of complexity for each
scenario, measured on a Likert scale from not at all complex (0) to very complex (4).
Table 3: Mean Level of Complexity by Scenario
Blackout
Mail
Train
Mean (n) Mode Mean (n) Mode Mean (n) Mode
2.8 (5)
2;4
3.0 (5)
3
3.2 (5)
4
Mean Level of Complexity
The results of the above table are also reflected in the average mean of the complexity factors,
with the Train Scenario having the highest mean complexity factor, followed by the Mail
Scenario and the Blackout Scenario. The mean of the impact factor was highest for the Train
Scenario. Uncertainty factor means varied little across the three scenarios, with the Blackout
Scenario having a slightly higher mean level of uncertainty. Likewise, vulnerability factor means
did not vary significantly, though the vulnerability mean was marginally higher in the Mail
Scenario and Train Scenario. The table below summarizes the key findings from the assessment
questionnaire on the factors of impact, uncertainty and vulnerability, providing the means and
modes separated by scenario.
Table 4: Means and Modes of Factors and Elements by Scenario
Blackout
Mean
Mode
(n)
Average Mean of Complexity
Factors
IMPACT
Impact Severity
Impact Scope
Impact Timing
Media Involvement
Political Processes
Mean of Impact Factor
UNCERTAINTY
Novelty of a Situation
Anticipation and Planning
Data and Information
'5'&&66&5
2.5
2.5 (6)
3.2 (6)
3.0 (5)
2.2 (5)
3.2 (5)
2.8
3.0 (6)
2.0 (6)
2.0 (5)
N/A
2
3
2;4
2
3;4
4
2
2
Mail
Mean
Mode
(n)
2.7
3.6 (7)
3.4 (7)
3.0 (7)
2.6 (7)
3.3 (7)
3.2
2.9 (7)
1.8 (5)
2.1 (7)
N/A
4
4
3
2
3
4
2
1;2
Train
Mean
Mode
(n)
2.8
3.9 (7)
3.9 (7)
3.6 (7)
3.4 (7)
3.6 (7)
3.7
2.4 (7)
2.5 (6)
2.4 (7)
N/A
4
4
4
4
4
2
2;3
2
19
New Organizations and Partners
2.4 (5)
Changing Context
3.0 (6)
Flexibility of Interpretive
2.4 (5)
Frameworks
Mean of Uncertainty Factor
2.5
VULNERABILITY
Economic Development
2.4 (5)
Social Capital
2.0 (6)
Community Competence
2.0 (5)
Information and Communication
2.0 (5)
Other Infrastructure
2.6 (5)
Mean of Vulnerability Factor
2.2
** Note: (n) is the number of item respondents
2
3
1.7 (6)
3.0 (7)
2
2;3;4
1.8 (6)
2.9 (7)
1
2
2
2.3 (6)
2
2.0 (7)
2
2.3
2
2
2
2;3
2;4
2.3
2.2 (6)
1;2
3.0 (7)
4
2.5 (6)
2
2.7 (7)
2; 3;4
2.3 (6)
2
2.5
2.2 (6)
2.6 (7)
2.0 (7)
2.9 (7)
2.9 (7)
2.5
1;2
3
2
4
2
Quantitative Analysis of the Results for the Blackout Scenario
Of the six reviewers who responded, half (50%) rated the overall scenario as complicated, where
the other half (50%) rated it as complex. Reviewers were asked which factor contributed most to
the level of complexity in the Blackout Scenario. Of the six reviewers who responded, the
majority (three) chose impact, two chose vulnerability, one chose all of the above, and none chose
uncertainty. This response was reflected in the means for each factor, whereby impact had the
highest mean (2.8), uncertainty had the second highest (2.5), and vulnerability had the lowest
mean at (2.2). Impact scope and media processes had the highest means among the elements
making up these factors.
Reviewers were also asked to rate their level of agreement with statements on the scenario’s
realism and plausibility. Of the four reviewers who responded, two reviewers were neutral and
two were in agreement (one agree, one strongly agree) with the statement that the first part of the
scenario was realistic and plausible. The majority (three) were neutral, and one agreed with the
statement that the second part of the scenario was realistic and plausible. Lastly, all four of the
reviewers who responded were neutral with the statement that the timeline part of the scenario
was realistic and plausible.
Quantitative Analysis of the Results for the Mail Scenario
Of the five reviewers who responded, the majority (60%) rated the Mail Scenario as complex and
40% rated it as complicated. Results were mixed concerning the reviewers’ estimations of which
factor contributed most to the level of complexity in the scenario. One reviewer stated that it was
impact, one stated uncertainty, and two stated vulnerability. Two reviewers reported all of the
above; indicating that they felt all three factors influenced the assessment equally.
Results were mixed in terms of questions on realism and plausibility. The majority agreed that the
first part of the scenario was realistic (two strongly agree, one agree and one neutral). One
reviewer strongly agreed that the second part of the scenario was realistic, where two were neutral
and one disagreed. Half of reviewers (two) strongly agreed that the timeline for the scenario was
realistic and the other half (two) disagreed with this statement.
20
'5'&&66&5
Quantitative Analysis of the Results for the Train Scenario
Of the six reviewers who responded, half (50%) rated the Train Scenario as complicated and the
other half (50%) rated it as complex. Four reviewers reported their estimation of which factors
contributed most to the level of complexity in the Mail Scenario. Two chose impact, one chose
uncertainty, and one chose vulnerability as contributing the most to complexity.
Nearly all reviewers (four) reported that they agreed that the first part of the scenario was realistic
(two strongly agree and two agree), while one was neutral. Two reviewers strongly agreed that
the second part of the scenario was realistic and three were neutral. Finally, three reviewers
agreed that the timeline for the scenario was realistic (one strongly agree and two agree), while
two remained neutral.
3.3.3.4
Summary of findings
Though reviewers found the train derailment scenario to be the most complex according to our
rating scales (albeit, not by a large margin), reviewers identified the radiological scenario as most
complex when asked during the verbal portion of the assessment process. Notably, prior to
completing the rating scales, reviewers most frequently cited the radiological scenario as most
complex when asked if the scenario was best described as simple, complicated or complex. After
completing the rating scales, reviewers cited the train derailment scenario as the most complex
when measured on a scale of not at all complex (0) to very complex (4). Corresponding results
between the rating scale means and overall complexity level means indicate some degree of
promise in the efficacy of the scales. The conflicting results may be partially explained by the
difficulty many have with distinguishing complicated situations from complex (as reflected by the
wording of the first question on describing the scenario as simple, complicated or complex).
Additionally, these conflicting results may indicate that reviewers are taking in more elements
into their decision after reviewing the scales than they did before filling out the rating scale.
The radiological scenario was subsequently chosen because of the number of organizations that
would become involved, locally, regionally, federally and internationally. In addition to being
highly complex, the scenario chosen also needed to plausibly involve the Canadian Forces; given
the security implications of a dirty bomb scenario, and the decontamination knowledge and
capabilities in the Canadian forces, it was determined that the radiological scenario best fit the
criteria.
Disagreements with respect to the timeline and plausibility of the scenario were reflected in the
discussion groups and in the questionnaire response, thus the research team concentrated its
efforts on scenario refinement in these areas. Additional attention was also given to making the
Mail Scenario even more complex.
3.3.4
Rating scale refinement for future research purposes
An extended version of the rating scale, complete with sub-elements, shows promise as a tool for
measuring situation complexity. This is indicated by results produced to-date. The rating scales
also have the potential to be extended for other uses. With further refinement and testing, these
'5'&&66&5
21
scales may be used in scenario development for training exercises and as a tool for retrospective
case study analyses.
3.4
Instrument development
A set of instruments was developed for the simulation experiment. These instruments were used
to measure the dependent variables, to provide background information on the participants, to
measure the participants’ ability to recall and understand simulation materials, and to measure the
quality of decisions made by participants. These instruments are described in more detail below.
3.4.1
Dependent measure instrument development
The dependent measures instrument (see Annex G) consists of three parts; the first part includes
questions on individual participation and perspectives. The second part is concerned with
questions on dynamics within the pods. The third part focuses on questions concerning dynamics
between pods. The third portion of the instrument was not included in the closed pod sessions.
Questions on these questionnaires are made to be answered on a five point Likert scale, from
strongly disagree (0) to strongly agree (4), where participants answer the level to which they
agree with the statement.
The questions measure participants’ satisfaction with problem solving process, level of
participation, task and group cohesion, and level of agreement. Questionnaires were reviewed
during initial pre-testing and pilot testing phases. It was discovered that a few of the questions
were duplicated on task one and task two questionnaires when they did not require duplication.
These questions were taken out of the task two questionnaires where they had been duplicated.
3.4.2
Background questionnaire
Participants were asked to complete a Background Questionnaire (see Annex H) during the
briefing session of the experiment. The questionnaire was comprised of both open and closed
questions. Participants were asked to provide their current job title, the length of time they have
worked in their position, with the organization and in emergency management, and to describe
their main responsibilities with respect to planning and responding to emergencies. Other sociodemographic information was collected such as their age, gender, language competency and level
of education. During the pilot testing, participants were assigned roles and thus filled out the
Background Questionnaire as actors. During the senior participant sessions, roles will be assigned
based on the participant’s current position and background.
3.4.3
Consistency of interpretation instrument
A consistency of interpretation questionnaire (see Annex L) was developed to determine the
degree to which the participants’ interpretation of the scenario matched the information delivered
during the session. The score on this questionnaire demonstrates whether the information
delivered to the participant was absorbed and retained. It may also reflect the participants’ level
of participation with respect to engaging with inject materials. The consistency of interpretation
instrument was designed to enquire about items of importance to emergency managers. Questions
22
'5'&&66&5
were specifically developed to ask about items that would be useful and important to carry out the
two tasks in the in vivo experiment. The questionnaire consists of ten questions and is
administered at the end of the session once all of the binders have been collected. It is distributed
at this time in order to ensure that participants are answering from memory and not from the
binder materials.
3.4.4
Decision quality rating instrument
One key dependent measure is the assessment of decision quality. The decisions developed by
pod participants via the coordinating and collaboration tasks were assessed by a small group of
raters (n=3) using a guide developed to collect their ratings on dimensions of decision quality
based on elements derived from decision analysis theory (Edwards, Miles & von Winterfeldt,
2007), and the Shared-Decision Making Framework. Examples of elements rated include use of
correct logic, indication of clear preferences, use of an appropriate frame, appropriate use of
resources, and creativity. The rating scale was developed using a 5-point Likert scale to capture
raters’ judgements on the various elements for each decision (see Annex N).
Given the diversity in expression of decisions across pod members and pods, the study team
formatted decisions using extractions from the individual worksheets and recordings so that each
decision rated was of similar format and had similar levels of detail. Once decisions were
compiled, they were randomly sorted for each rater, with the rater remaining blind as to whether
the decision had been derived from a collaboration or coordination task. A rating guide was
completed by each rater for each decision. All decisions from communication tasks were grouped
together for assessment, as were the decisions from the health and safety tasks. Inter-rater
reliability was assessed using Fleiss’s kappa and intra-class correlation.
3.5
Development and assessment of simulation tasks
The development of the simulation tasks began prior to the scenario assessment process. Draft
versions of these tasks were assessed during scenario rating sessions and individual meetings with
expert raters. Following the scenario selection process, the radiological scenario was developed
for the simulation, and the tasks were further refined to reflect specific issues of concern brought
out by this scenario, such as health and safety concerns. This section of the report describes the
links between the simulation tasks and the Shared Decision-Making Model, the rationale for
assessing simulation tasks, and the task development and refinement process.
3.5.1
The Shared Decision-Making Model as it relates to simulation
tasks
The experimental design involves the manipulation of two levels of the independent variable,
approach to problem-solving. These levels are coordination and collaboration. These two levels
are depicted on the right hand side of the Model for Inter-Organizational Problem Solving, which
is pictured and described in the introduction section of this report. In order to manipulate the
implementation of these problem-solving approaches, two sets of tasks were developed – one set
of tasks to elicit coordinating behaviour, and one set of tasks to elicit collaborating behaviour.
'5'&&66&5
23
In addition to facilitating the manipulation of the approach to problem solving variable, the tasks
were also designed to limit the moderating effects of problem solving stage to three stages of the
problem solving cycle represented in the figure below. Each task was divided into three sub-tasks,
and each of these sub-tasks related directly to one of the following problem-solving stages:
problem definition, solution generation, and decision making.
Figure 5: Generic stages of problem-solving
3.5.2
Rationale for assessing simulation tasks
Draft tasks were reviewed by individuals with an expert level of knowledge in the field of
emergency management, and who were currently members of organizations involved in
emergency response. The assessment of these tasks was completed through meetings and sessions
held with these reviewers. In particular, these assessments sought feedback with respect to
relevancy in terms of the problem identified in the task. Validation was also necessary in order to
ensure that the tasks elicited the desired behaviour (coordinating or collaborating) from the
participants. Reviewers agreed that the tasks and task worksheets elicited the desired behaviour.
Suggestions were made by the reviewers to modify some issues identified in the tasks.
3.5.3
Task development and refinement process
The final tasks were developed to reflect a number of design considerations. These considerations
required that the tasks:
• Elicit the desired behaviour and approach to problem solving (coordinating, collaborating)
• Include three sub-tasks that replicate the three problem solving stages targeted in the
experimental design (problem definition, solution generation, and decision making)
24
'5'&&66&5
• Be completed within 45 minutes to 60 minutes
• Address issues relevant to the participants
• Be plausible within the selected scenario
Initially, both tasks focused on communication issues. Expert reviewers suggested that one task
focus on either evacuation or on health and safety issues of responders. Health and safety was
later chosen following the selection of the radiological scenario, as this theme is both relevant to
study participants and plausible within the scenario.
The communication tasks and the health and safety tasks were structured using task worksheets,
which were completed by participants (see Annex K). These task worksheets were developed and
worded differently to elicit either coordinating behaviour or collaborating behaviour. Task
worksheets were also adapted for closed pods in order to reflect the lack of inter-pod
communication within these pods.
3.6
Development of multimedia simulation
Following the scenario and task selection and refinement process, the mail scenario was then
transformed into a multimedia simulation using video, animation, audio and slide presentation
software. The outputs from the multimedia software were augmented by paper-based materials as
well, which were included in the participant materials distributed at the start of each session. This
section includes the design considerations for the multimedia components, the techniques used to
create these components, and finally, a description of the multimedia components used in the
Inter-GAP In Vivo System.
3.6.1
Design considerations
While the scenario used during the selection and refinement process was designed to be as
detailed as possible to ensure that the raters were able to assess complexity, some details were
removed in the final multimedia version of the scenario. This was particularly the case with
technical aspects of the scenario that could be considered potential areas for debate and
contention among participants (e.g., the first responder group to arrive on scene). To ensure that
the participants could fully engage in the scenario without debating unnecessary details, the
scenario was reviewed and revised to include only those details which potentially would have a
direct bearing on the tasks required.
The introduction of situation reports and news reports into the multimedia components served to
encourage participants to fully engage with inject materials. It was determined that, by
introducing familiar formats of information such as situation reports, news reports and press
releases into the simulation materials, participants would become more fully immersed in the
scenario.
Social media outputs were also included among the materials that participants received in the
final version of the mail scenario. Social media components included a YouTube clip, a series of
Facebook posts, as well as a number of Twitter feeds (see Annex O for Pods Participant
Workbook Materials). These injects were used to add complexity to the types and amounts of
'5'&&66&5
25
information that the participant received. It required participants to sort through the information
for relevancy and reliability, much as responders are required to do during extreme events.
3.6.2
Creating the simulation
After undergoing validation by a series of experts in emergency management, as well as by a
number of radiological and military experts, the ‘dirty bomb’ mail scenario was transformed into
multimedia components. Video clips were completed using Xtranormal animation software,
Adobe Premiere, and stock video footage. Social media feeds and posts were created using
Facebook and Twitter, as well as with Adobe Photoshop. The fictional city of Gapville, used to
situate the scenario, was created using Adobe Illustrator. Individual Gapville maps, which were
part of the participant materials handed out during the simulation, were also created with the aid
of Adobe Photoshop and Adobe Illustrator. Microsoft Word was used to create the Public Health
Agency Press Release inject, while PowerPoint was used to create the instructions for the
participants that were delivered via NEFSIS video teleconferencing software.
3.6.3
Multimedia components
The multimedia components are described in this section. These components include video clips,
the illustrated fictitious city of Gapville, paper-based injects, and PowerPoint presentations.
Video Clips
Video clips were created to simulate a report from an emergency operations centre (EOC), to
simulate news reports and updates, and to simulate a YouTube rant from a disgruntled member of
the public. The audio tracks, watermarks and the ticker tape information running across the
bottom of the screen were added to the videos using Adobe Premiere. Some of the clips were
augmented with stock video footage that was purchased for a nominal fee through iStockphoto.
Video clips were animated using Xtranormal Technology’s video software. Voice actors were
used to create the voices for animated avatars, which took the place of live actors in the clips. The
voice actors were casted from volunteers from the University of Ottawa’s theatre program.
Xtranormal’s avatars allowed the research team to create a scenario that is both realistic and yet
sufficiently removed from reality (in that the characters are animated rather than live actors) to
reduce the potential for psychological stress in participants. This software also allowed a high
degree of flexibility during the design phase. Its use was both more efficient and more costeffective when compared with live action video methods. See Figure 6 below for screenshots
from the simulation.
26
'5'&&66&5
Figure 6: Screenshots from the Inter-GAP In Vivo System scenario injects
Gapville
The mail scenario is situated in the fictional city of Gapville, a highly detailed two-dimensional
rendering of a Canadian city located on the border to the United States. This training tool was
created as part of the Psychosocial Risk Manager (PRiMer) training program, funded by the
Centre for Security Sciences, CRTI-06-0259TD project. Created for use in learning activities and
tabletop exercises, this cityscape illustration contains all the built and social infrastructure of a
mid-sized North American city. Additionally, the city of Gapville was created to bring
psychosocial issues to the forefront of emergency management training exercises, with a diverse
range of at-risk populations depicted alongside other populations that require psychosocial
consideration. A visual representation of the city of Gapville can be seen in Figure 7.
'5'&&66&5
27
Figure 7: Simulated city of Gapville utilized in the Inter-GAP In Vivo System
Paper-based Injects
A press release from the Public Health Agency of Canada, individual maps of Gapville, and
outputs from simulated Twitter and Facebook posts or feeds were all presented in a paper-based
format. These injects were included as part of the package of information given to each
participant at the start of the simulation session.
PowerPoint Presentations
A PowerPoint presentation was used to deliver the instructions to participants over the NEFSIS
video conferencing system. Given that instructions varied slightly from the open pods to the
closed pods, two versions of this presentation were created. The slides used in these presentations
can be seen in Annex P.
3.7
PODS session roles, responsibilities and materials
This section describes POD session roles, responsibilities and associated materials used by the
facilitator, controller, observers and participants during the course of a complete session of the
simulation. Many of the specific forms and instruments used in the simulation have been
described already in earlier sections of this report and are only referenced here as appropriate.
References are made to both electronic media and paper-based materials but only copies of paperbased materials are included in the appendices.
28
'5'&&66&5
3.7.1
Facilitator role and materials
Facilitator role
The facilitator’s role was to oversee the proper conduct of each session of the experiment as per
ethics requirements. The facilitator held a briefing session prior to the experiment followed by a
debriefing session at the end of the simulation. The facilitator provided participants with the
study’s agenda, as well as directions on how to complete the tasks. An explanation of the study’s
purpose and background was explained in more detail following the completion of the study. The
facilitator ensured that all forms and materials were properly administered, completed and
collected throughout the day.
Briefing materials
The following briefing materials were used during the briefing session prior to the start of the
experiment:
• Information Sheet & Consent Form (Refer to Annex O & F)
• Confidentiality and Intellectual Property Agreement (Refer to Annex I)
• Participant Background Questionnaire (Refer to Annex H)
• A short introductory presentation, welcoming participants, explaining their role and
communication options within the pods (Refer to Annex P.1 for presentation slide deck.)
The briefing session lasted about half an hour prior to study commencement. Participants were
invited to fill out consent and confidentiality forms as per ethics requirements. They were
informed of their voluntary participation as well as anonymity procedures. Each participant also
filled out a Participant Background Questionnaire. Briefing participants on the schedule and
directions for the experiment allowed for questions and concerns to be addressed by the
facilitator. This ensured transparency of participant roles and responsibilities expected during the
experiment.
Upon completion of the briefing, participants were assigned to a POD and taken to separate
rooms. When in the separate rooms, each participant received a binder with more background
information about the experiment and their role. In line with anonymity and confidentially ethics
guidelines, participants were assigned a number when given their participant workbook.
Participants were then instructed to identify themselves on video by the number assigned them for
tracking purposes. A hardcopy of the Gapville map was provided to each POD member with a
Gapville profile and demographics table inserted into their participant workbook. The complete
contents and structure of the participant workbook are described below in Section 3.7.4
Participant Materials. A short video depicting the Gapville profile and its background
information was sent to each pod via video conferencing software (refer to Section 3.6
Development of Multimedia Simulation for more information about the multimedia components
of the simulation).
Debriefing materials
The following materials were used during the debriefing session:
• Consistency of Interpretation Instrument (Refer to Annex L)
'5'&&66&5
29
• Two video clips - an overview of the Model of inter-organizational problem solving and a
summary of the research objectives and experimental design.
The debriefing session lasted half an hour. Participants were asked to complete the Consistency of
Interpretation instrument. The facilitator then showed participants the two video clips.
Participants provided feedback and were able to ask questions regarding the study. The facilitator
ensured that all forms and binders were handed in.
3.7.2
Controller role and materials
Controller role
The controller was responsible for running the video conferencing software and guiding the
participants as the scenario unfolded. The controller warned participants of time limits on tasks,
informed them when communication between PODS was open or closed and directed them to the
Nefsis chat function and CB radios.
Simulation delivery script
The controller referred to a script (see Annex P.2) outlining the duties and procedures for
delivering the simulation. These duties related to technical issues (e.g., plugging in headsets,
recording, audio adjustments, etc.), sequencing media delivery (e.g., scenario videos, power point
slides, etc.), layout (i.e. the way participants viewed themselves in the conferencing software and
the size of the slideshow and video feeds), and voice over script for time warnings and
instructions. Included in this script was a breakdown of activities, including the time and slide
that belonged with them.
3.7.3
Observer role and materials
During each pilot session, a number of observers were present. The observer’s role was created to
oversee the distribution and collection of participant materials and forms, as well as to ensure the
linking of participants’ identification numbers with the written materials, forms, and audio and
video recordings from the in vivo sessions.
Prior to the start of each session, questionnaires and worksheets included in the participant
binders were labelled with participant ID numbers using “sticky labels”. During the initial
briefing, as participants handed in their consent forms, they received a binder of participant
information and questionnaires labelled with their participant ID number. Four loose identifier
labels located at the front of the binder were removed as each participant received their binder,
which were then affixed to the corresponding forms for that participant. In order to identify
participants with the audio and video recordings, participants were asked to address both the
video camera and the web camera and state their name, organization, and participant ID number
prior to the start of each session. These statements were made following the briefing session, once
participants had settled into their separate pod rooms, but prior to the start of the simulation
30
'5'&&66&5
3.7.3.1
Observation forms
During each session, a number of forms were completed by the observers to track administrative
information and observations about each session, pod and participant. The three forms developed
and implemented were:
• Session Administration Information Form
• Participant Identification and Information Linking Form
• Technical and Process Issues Form
Session Administration Information Form
This form was completed for each session (one form per session). Once the briefing session had
been completed and the simulation was underway, an observer was tasked with capturing any
information that could not be completed prior to the start of the session (e.g., number of
participants present, number of pods, etc.). It was used to record basic information about the
session, including: date, session number, session type (university students, professional students,
or senior officials), task type (collaboration or coordination), pod status (connected or closed),
number of participants per pod, pod type (mixed or homogeneous), scenario used, location, the
observer’s initials, the facilitator, the start time and end time of the session, as well as the total
runtime for the session. This form was then used to link the data for these variables to each
participant record in the SPSS database, where appropriate. (Please see Annex P.3 for a copy of
the Session Administration Information Form.)
Participant Identification and Information Linking Form
This form was completed for each participant (one form per participant). Once the briefing
session had been completed and the simulation was underway, an observer was provided with the
completed participant consent forms collected during the briefing phase, and was tasked with
completing the relevant participant form (matched according to labels on the Consent Form, the
Confidentiality and Intellectual Property Agreement Form, the Background Questionnaire, as
well as the Participant Identification and Information Linking Form). The Participant
Identification and Information Linking Form was designed to capture the following participant
information: participant’s name, the four digit participant ID, the session number, session type
(University of Ottawa students, professional students or senior officials), the participant’s pod
number, the participant’s number within the pod, the participant’s organization, the participant’s
organization type (military, ICS oriented, and non-ICS oriented), the participant’s pod status
(connected or closed), as well as the number of participants in the pod. (Please see Annex J for a
copy of the Participant Identification and Linking Form.)
Technical and Process Issues Form
This form was completed for each session (one form per session). Observation forms were used
during the sessions to record any technical or process issues that had the potential to affect the
data quality of the session. The technical issues form consisted of questions regarding the
presence of audio or video delivery problems, audio or video recording problems, inter-pod
communication problems due to technology, conferencing software problems, internet connection
failures, and a section for other unanticipated technical issues. The process issues form was
'5'&&66&5
31
concerned with the following problems: participant absenteeism, participant lateness, participants
leaving early, consent refusal, delayed start time, session interruption, participant confusion, and
other unanticipated process issues. In both the technical issues form and the process issues form,
there were prompts to describe the nature of the problem(s) in more detail. (Please see Annex P.4
for a copy of the Technical and Process Issues Form.)
3.7.4
Participant materials
Participant workbooks were prepared for use by participants for each of the four types of sessions
conducted. Tasks, questionnaires and worksheets provided to participants varied slightly
depending on the task type (coordination or collaboration) and the pod status (open or closed).
Below are the four types of participant materials:
1.
Open / Coordination
2.
Open / Collaboration
3.
Closed / Coordination
4.
Closed / Collaboration
Refer to Section 3.5 for a detailed description of the types of tasks. Annex G provides examples of
the questionnaires differentiated by task type and pod status. Annex K provides examples of the
task worksheets differentiated by task type and pod status.
The briefing materials contained in the workbooks are similar across all four types of sessions.
Also similar across all four types of sessions are the hard copy versions of the situation reports
and associated press releases and social media feeds. A debriefing sheet was provided containing
the contact information of the research team should participants have any questions about the
simulation at a later date. The sequence of material found within each participant workbook is
listed below.
• Gapville Profile
• Situation Report #1+ Public Health Agency of Gapville Press Release Inject *
• Task 1 + Task 1 Questionnaire *
• Situation Report #2 + Facebook Post Inject + Twitter Feeds Inject *
• Task 2 + Task 2 Questionnaire*
• Debriefing Page
Materials marked with an asterisk were put into separate envelopes within each participant binder
and opened as per simulation procedure. Annex O contains an example of only similar materials
across all session types. Annex K contains the different task types and Annex G the different types
of task questionnaires.
32
'5'&&66&5
3.7.5
Technical and logistical support
On the day of each session, a technician was responsible for setting up each room as per section
3.9 of this report. The technician ensured that the POD rooms and the control room were properly
set up and that the video conferencing software was functioning properly and accessible. This
person was on hand throughout each session to monitor in case any problems occurred. Logistical
support included arranging refreshments for participants and making sure appropriate signage was
put up to direct participants to the study.
3.8
Recruitment
Recruitment efforts centred on finding three types of participants: experts for scenario and task
assessment as well as for decision quality rating; naïve participants and junior professionals for
pilot testing; and senior level emergency management professionals for the full scale experiment.
Participant involvement in the study was completely voluntary and consent was obtained from
each participant as per ethics requirements.
Recruitment of Experts
Experts were needed for the assessment of the tasks and scenarios, for the senior level re-review
session, and for the decision quality rating session. Nine experts were recruited for the scenario
and task assessments. These experts provided valuable feedback on the accuracy, plausibility and
complexity levels of the scenarios drafted for the scenario selection process. Expert participants
were required to be willing to participate in the study, able to participate in English, and able to
attend scheduled meetings or sessions. An expert level of knowledge of emergency management
and response to extreme events was also required. See the table below for specific inclusion and
exclusion criteria for this group.
Table 5: Inclusion and Exclusion Criteria for Expert Recruitment
Scenario and Task
Assessments
Senior
Level
Demo
Decision
Quality
Rating
Expert knowledge of emergency management and response to extreme
events
9
9
9
Able and willing to volunteer to participate in the study
9
9
9
Able and willing to participate in English
9
9
9
9
9
Inclusion and Exclusion Criteria
Outside commuting distance of the study site
Member of an organization involved with emergency response or
management
Has been in management-level decision making role for his/her
organization for at least 12 months (or has was in a management-level
decision making role for his/her organization for at least 12 months
prior to occupying a new position or prior to retirement)
'5'&&66&5
9
9
9
33
Recruitment for Pre-testing and Pilot Sessions
Naïve participants were required for the pre-testing and pilot testing sessions. Students and naïve
participant volunteers provided valuable feedback related to the content and process of the
simulation. Junior level career professionals and students engaged in emergency management
programs, the military and non-governmental organizations were targeted for the junior level pilot
sessions. These sessions provided valuable feedback on the instruments used in the experiment,
on session materials and on the development of coding schemes. Inclusion and exclusion criteria
for this groups of participants is shown below.
Table 6: Inclusion and Exclusion Criteria for Pre-testing and Pilot Sessions
Inclusion and Exclusion Criteria
Naïve
Internal
Participant
Pre-testing
Pilot
Sessions
Sessions
Professional
Students/
Junior
Career
Sessions
Able and willing to volunteer to participate in the study
9
9
9
Able and willing to participate in English
9
9
9
Outside commuting distance of the study site
9
9
9
Student, volunteer or team member
9
University student or volunteer naïve to the purpose of the
study
A student in a professional emergency management program or
a junior level member of an organization involved with
emergency response (a member a military, ICS, or a relevant
non-governmental organization)
9
9
The research team used a University of Ottawa study recruitment program, called the Psychology
Integrated System of Participation in Research (ISPR), to recruit naïve participants during the
pilot testing phase. The present study was posted on this system and time slots were made
available to first year psychology students. Refer to Annex Q.1 for a screenshot of the posting
made on the ISPR system. The research team was notified by email when a student signed up for
a time slot. Students remained anonymous and only an identification number was provided. Upon
completion of the study, the research assistant entered the ISPR system and awarded the student
their participation credits.
During the pilot testing phase, naïve participants were assigned fictitious roles within the
scenario, as preliminary testing indicated that the limited knowledge that these participants
possessed of emergency management prevented the participants from fully engaging in the
scenario. Roles assigned to these participants included:
34
'5'&&66&5
• Head of the Red Cross (National)
• Gapville Fire Department Chief (Local)
• Colonel of the Reframe Military (Federal)
• Mayor of Gapville (Local)
• Head of Public Health Agency (Federal)
• Anticipate Regional Transport Authority (Provincial)
• RCMP Lead Investigator (Federal)
• Chief of the Gapville Police (Local)
• Director of the McLaughlin Memorial Hospital (Local)
• Head of Public Safety (Federal)
Recruitment for Full Sessions
Forty-eight senior level decision makers are targeted for the four full sessions scheduled to occur
over April and May of this year. These senior level decision makers will be members of ICS, nonICS and military organizations. See Table 7 for a summary of the participants required to
complete the PODS experiment.
Table 7: Summary of Participants Required to Complete the PODS Experiment
Phase of Session
Full Sessions
(3 pods plus mixed
closed/ substitute pod)
# Total
# Military # ICS # Non-ICS # Total
4
4
4
12
4
4
4
12
4
4
4
12
4
4
4
12
16
16
16
48
Full sessions will require participants with a minimum of one year experience in a management
role in response to extreme events, previous experience with at least one extreme event, and
decision making authority within their respective organization. Inclusion and exclusion criteria
for full session recruitment sessions are shown below.
Table 8: Inclusion and Exclusion Criteria for Full Sessions
'5'&&66&5
35
Inclusion and Exclusion Criteria
Full Sessions
Previous experience in both strategic and operational decision-making during at least
one major event
9
Has been in management-level decision making role for his/her organization for at least
12 months (or has was in a management-level decision making role for his/her
organization for at least 12 months prior to occupying a new position or prior to
retirement)
9
Has the authority (or has had the authority in the past) to make decisions regarding
allocation of resources on behalf of their organization with respect to major events.
9
Able and willing to volunteer to participate in the study
9
Able and willing to participate in English
9
Outside commuting distance of the study site
9
3.8.1
Communication Process
In order to organize the various participants required for completion of the experiment, steps were
outlined for the recruitment process and communication documents were drafted. A formal
process was developed for the recruitment of participants who were not solicited through the
University of Ottawa’s ISPR system. Expert raters contacted during the scenario and task
assessment phases and those who will be contacted for the decision quality rating phase of the
project may be completed via telephone as participant availability permits. The formal
recruitment protocol for the remainder of the participants in the PODS project is as follows:
• Step 1: Review existing database of contacts
Existing contacts were reviewed and participants were selected that were likely to meet the
inclusion criteria.
• Step 2: Request additional contacts
Additional contacts were requested from experts consulted during the scenario and task
assessment processes. Senior officials at the Canadian Forces were also contacted via e-mail
in order to gain assistance with recruiting military personnel.
• Step 3: Invite participants
Participants were invited via e-mail to take part in the simulations. In some instances,
participants were contacted by phone first. An invitation letter was drafted both for
organizations and for individuals (see Annex E & Q.2).
• Step 4: Confirm that interested participants meet inclusion criteria and determine
availability
Phone confirmation was used to find out if interested participants met inclusion criteria, and
to determine participant availability. A phone script was used during the confirmation
process (see Annex Q.3). Follow-up occurred by phone or e-mail as required.
36
'5'&&66&5
• Step 5: E-mail reminder
Participants were sent an e-mail reminder (see Annex Q.4) prior to the session dates. A brief
letter was written which included the time, location, date, directions, parking information
and contact information. Participants were asked to contact the research team should they
need to cancel.
• Step 6: Send a thank you letter
Thank you letters were sent to session participants within one week of the completion of
each session. These thank you letters, sent via e-mail, thanked participants for their
contribution to the project (see Annex Q.5 & Q.6 for copies of these letters).
3.9
Equipment, facilities, software, data collection and storage
This section provides a summary of the equipment, facilities and software choices made for the in
vivo simulation exercise. (Refer to Annex R for a complete equipment and room requirements
listing.) Preliminary equipment, facilities and software needs were documented in the
experimental plan developed under Task 2. However, as the hardware and facilities testing took
place before and during the participant pilot tests, these requirements were modified.
The section begins with a discussion on the use of video conferencing software for the experiment
before describing the type, extent and outcomes of the technical testing conducted prior to the
start of the formal experimental sessions with senior decision-makers. Please note that Section
3.10 Pilot Testing also contains summary comments related to technical and process issues
encountered during the participant pilot sessions. These comments were documented by assigned
observers for each of the pilot sessions and are not repeated here. The section ends with a brief
description of the final hardware requirements and room configurations for the experiment and
electronic data collection and storage protocols.
3.9.1
Rationale for use of conference software
The in vivo experiment required a system for delivering simulated data simultaneously to a
variable number of pods. It also required an efficient method of data capture for text, video and
audio. This was particularly important given the amount of data to be collected over the course of
the experiment and the multiple levels of analyses to be performed.
Commercially available video conferencing software proved to be an ideal solution to meeting
these needs. It offered a robust study environment where the sessions could be served locally (i.e.,
to designated meeting rooms or working areas) or remotely, to participants joining in from homes
and other places of work. It offered different propriety systems of data collection, including video
and chat recording. With online video conferencing software, participants can see and
communicate with members of other pods. Although body language and expression can be very
subtle, it is a powerful means of communication and understanding. The opportunity to review
and analyze such data will inform the findings of the experiment. Also, it proved to be less
expensive to run the sessions through the internet (be it locally or remotely) than it would be to
hard line all the necessary equipment required to implement the same experiment with more
traditional technologies.
'5'&&66&5
37
Implementing an online video conferencing solution made it possible to create a functioning
Hydra inspired system of disseminating and collecting data within a reasonably limited budget. It
was also hoped that by using online software the experiment could be run remotely, if necessary.
3.9.2
Software selection process
A number of online video conferencing software systems were considered for the experiment,
including NEFSIS, GoToMeeting, NetMeeting, Adobe Connect, DimDim and WebEx. While
there was overlap in terms of benefits and features for each of the systems reviewed, the NEFSIS
system came closest to meeting our overall requirements. These benefits and features included:
• Ease of use for participants
It was important that participants be able to easily and comfortably communicate between
pods. NEFSIS offered both an intuitive and customisable interface.
• Ability to record meeting sessions
Of the platforms explored, NEFSIS was the only one offering video recording of meetings
as a function. With this function, sessions can be recorded with full audio and video. This
was an important factor in choosing a meeting platform, as it facilitated the data collection
process for multiple media types.
• Ability to capture chat messaging
Most of the platforms reviewed offered methods of saving chat messages. However,
NEFSIS and Goto Meeting were the only two that offered simple one click solutions.
NEFSIS provides a chat window allowing participants to communicate in a free bidirectional manner. It enables the saving of all chat files as '.txt' files. This means that all
chat messaging done during the sessions can be captured for later review and content
analysis.
• Security
Due to ethical considerations, meetings held as part of the experiment had to be held
securely. Additionally, certain materials needed to be held back from participant until it was
time to deliver them as scenario injects. These materials remained hidden until they were
delivered according to a pre-determined schedule. NEFSIS facilitated these needs through
strict user access controls. Administrator privileges can only be given through a confidential
login or directly from another administrator, meaning that participants would be unable to
access any information they are not given. Additionally, these system controls ensure that
the system remains secure from individuals who are not invited to participate in the
meetings.
• Ability to deliver information in multiple forms
NEFSIS allows for multiple forms of information sharing and exchanging. Among the
options are screen, application and multimedia file sharing. For the purposes of the in vivo
experiment, NEFSIS injects were delivered exclusively via PowerPoint or multimedia
video. The simulation included simulated video news reports as well as instructions
delivered with on screen PowerPoint slides.
38
'5'&&66&5
• Ability to deliver smooth audio/video files
The use of different types of media during the simulation created the potential for long
pauses and poor transitions from one simulation segment to another. To limit visual and
audio distractions and keep participants engaged, videos needed to be smooth and free of
pauses and glitches. NEFSIS was able to accommodate this requirement.
3.9.3
3.9.3.1
Technical testing of equipment, facilities, and simulation delivery
system
Technical testing and equipment and facilities re-configuration
Technical tests were done on specific equipment and software throughout the fall, prior to the
first pilot test with participants. The purpose of these tests was to ensure the proper functioning
and set-up of the individual equipment / software components of the simulation system. In
addition to these technical tests, three technical run-throughs of the experiment were conducted
under “full load” conditions. For these tests, all required hardware and rooms were set-up and the
complete simulation delivered by video conferencing software. Several of the tests involved
practice delivery and reception of the simulation from remote locations.
The nine pilot tests with participants also offered additional opportunities to debug
hardware/software issues and modify equipment settings and location for optimal performance.
Most of the pilot tests were conducted under “steady-state’ conditions, i.e., after all equipment
and location choices were finalized, as well as simulation content, sequence and processes.
The initial concept for the simulation exercise was based on the best practices of several different
systems used across Canada, and worldwide in operational training exercises. The simulation
system developed at GAP-Santé, now known as the Inter-GAP Simulation System was intended
for deployment at an external off-campus location with potential delivery to various selected sites
across the country. It was anticipated that this would help facilitate recruitment of senior decision
makers who are geographically dispersed across Canada. It would increase the availability of
participants and reduce travel time and associated expenses. In order to create this set up, the
Inter-GAP System required 18 laptops. Each participant in a session was to have their own laptop,
pre-loaded, for a possible 12 computers needed for participants. In addition two laptops are
needed for the controllers (one for open pod delivery and one for closed pod delivery) and four
laptops are needed for observers. Routers and switchers were also obtained for splitting internet
connections at an unknown or remote location. Printers were also purchased for document sharing
or inject delivery.
Upon initial testing it became apparent that wireless internet connections would not have
sufficient bandwidth to deliver the media rich injects provided in the scenario. It also became
apparent that a technician would need to be on stand-by at any remote location in order to help
the user download the software, adjust their speakers and microphones, and ensure they were on
camera, and able to hear and see properly. This negated the feasibility of having pods in remote
locations.
It was decided then to locate the experiment entirely at University of Ottawa facilities. During the
technical run-throughs for a single pod using University of Ottawa student volunteers, it was
'5'&&66&5
39
noted that immediately after a simulation inject was delivered, participants moved away from
their computer screens (and web cams) to sit at a table together. In an attempt to accommodate
this demonstrated behaviour, and to have a more realistic experience for the participant, each
room was re-configured with only one computer and a large monitor. All three participants in a
pod now shared the same work table and viewed the same monitor. The use of a single computer
per room was also supported by the fact that most meeting rooms are configured at the University
for only one internet connection. The set-up of multiple connections per room posed logistical
and security implementation problems. With all the pods in close proximity to one another, the
research team also decided to introduce the use of two-way radios (one per room) as an
alternative communication vehicle between pods to add realism to the simulation.
3.9.3.2
Technical testing of simulation system – summary of solutions
As alluded to above, the overall configuration of the pods evolved over time as the simulation
system was tested in different circumstances and environments. In testing the simulation system,
there were two main technical issues to refine. They were:
Audio/video quality:
Each meeting room required clear audio and video communication between the pods and with the
control room. Without clear audio, pods would not be able to work together easily, and without
clear audio to the control room, data recordings could not be reviewed easily for analysis. In early
technical run-throughs, it was determined that while the video quality of the cameras was
sufficient, the audio was less than satisfactory. In these early sessions the audio microphone was
integrated into the web camera. Unfortunately, these camera microphones created significant
feedback issues when used in conjunction with the video camera, computer speakers and MP3
voice recorders. This issue was later solved when USB conferencing speaker/microphones were
utilised. The new microphone/speakers featured integrated technology to cancel out feedback
created from other sources.
Internet connection quality:
PODS sessions were run with both the control room and pods on and off campus. The system was
also tested to compare its wireless performance to hard-wired performance.
When the experiment was run with computers on a wireless connection, there was a noticeable
lag in communication between pods and delivery of injects from the control room. Some
participants also lost their wireless connections and were occasionally ejected from the session.
The session configuration was also tested remotely using traditional hard line internet connections
from various locations throughout Ottawa. Meetings run in this fashion worked, however video
injects sent via the control room buffered more slowly and their display was considered to look
somewhat choppy.
40
'5'&&66&5
3.9.4
Equipment and facilities requirements and set-up
The following section reflects the specific equipment and set-up required to run the Inter-GAP
Simulation System. For an in depth description of the required equipment and set-up, see Annex R.
Control Room
The control room houses two computers each with a dedicated internet connection. One computer
is used for running the open pods and one computer is used for running the closed pods. Each
computer must be connected to a different NEFSIS meeting room and have a
microphone/earphone headset. These computers both need to be pre-loaded with the videos and
power point presentation needed to complete the sessions. During each session the meeting video
and bi-directional chat is recorded, collected, and later stored in a secure location. Extra seating is
also available for observers.
PODS Rooms
Each pod requires a hard-lined internet connection. Rooms must also be sufficiently large enough
to contain all the pod equipment and the three participants. Additionally, there must be sufficient
room such that a video camera can be placed far enough away from the participants to capture all
of them in the camera frame. See Figure 8 for a typical pods configuration.
Figure 8: A typical POD room configuration.
Open pods are set up with one computer in each room. A web camera, and a microphone/speaker
speaker unit are connected to each of these computers. The computers are connected to the
NEFSIS meeting room prior to the start of the session. Additionally, a video camera is set up
behind each computer to capture and record all the events within the room. These rooms are also
equipped with a two-way radio and a voice recorder.
Closed pods are isolated from the other pods. Consequently, they do not require a microphone or
web cam. Instead closed pods are set up with one computer, one voice recorder, and one video
camera only.
'5'&&66&5
41
Brief/Debrief Rooms
Brief and debrief rooms need to be large enough to comfortably fit up to 18 people. These rooms
should also have live power outlets to run a computer and multimedia projector.
3.9.5
NEFSIS set-up and administration
NEFSIS meetings were initiated by logging on to the appropriate meeting through an assigned
web address. After entering the meeting room, administrators adjusted the privileges for each
pod, such that participants were not able take over use of the screen, nor were participants able to
present materials. The administrator also configured the audio and video settings, and pre-loaded
all scenario inject materials. Finally, administrators initiated the recording function prior to the
start of each meeting.
3.9.6
Data collection and storage
After the completion of each PODS session, all of the digital recording files were collected. These
files included: meeting chat files, NEFSIS video recordings, video camera recordings and digital
voice recordings. All files were then uploaded to an encrypted external hard drive. Materials were
named using the session date, media type, and pod number. A backup of all materials was also
made on a second external hard drive, which was also stored in secure location. During the
analysis phase of the project, materials will be uploaded to a secure password protected folder on
the network drive for the research unit.
3.10 Pilot testing
3.10.1
Overview of the pilot testing
Pilot testing consisted of having various senior managers’ help with the selection and rating of
potential scenarios. This was followed by internal pilot testing, pilot testing with naive
participants, and pilot testing with professional student/early career participants. Pilot testing
concluded with a senior level demonstration and review (see Figure 9). Pilot testing consisted of
ten sessions, which began in November of 2010 and was completed in March of 2011. These
sessions varied by type (coordination and collaboration, open and closed), number of participants
and number of pods (see Table 9).
42
'5'&&66&5
Figure 9: Review and Pretesting Stages
'5'&&66&5
43
Table 9: Overview of the Pilot Tests
Session 01
Uottawa students
Tues-Nov 30
Collaboration
Closed
3
Number
of
PODS
1
Session 02
Uottawa students
Wed-Dec 1
Collaboration
Open
9
3
Session 03
Uottawa students
Thurs-Dec 2
Collaboration
Open
9
3
Session 04
Uottawa students
Fri - Jan 21
Coordination
Open
4
2
Session 05
Uottawa students
Mon - Jan 24
Collaboration
Open
5
2
Session 06
Professional program
Fri- Jan 28
Coordination
Open
5
2
Session 07
Professional program
Thurs-Feb 3
Collaboration
Open
8
3
Session 08
Professional program
Thurs- Feb 24
Coordination
Open
10
3
Session 09
Professional program
Fri- Feb 25
Collaboration
Open
10
3
Session 10
Senior officials "demo"
Fri - March 18th
Collaboration
Open
TBD
TBD
Session #
3.10.2
Participant Group
Date
Number of
participants
Type
Pilot testing results
The following section presents the preliminary results of the pilot testing phase of the PODS
project. Topics addressed include the content and process results of the pre-testing and pilot
testing phases, the results from the senior demonstration session, and the preliminary quantitative
results of the selected pilot testing sessions.
3.10.2.1
Content and process
Pilot testing sessions yielded important information concerning required content and process
changes in the in-vivo simulation experiment.
Content
The PODS project simulation materials were based upon training exercises used before the
Vancouver Winter Olympic Games. Those materials were delivered in the form of a live action
news broadcast, with field reporters. Using the radiological event scenario, the simulation
materials in the PODS project uses digital avatars to deliver news broadcasts with field reports.
Due to the stringent selection, and intensive subject-matter expert reviews during the scriptwriting phase of the scenario, there were very few changes to be made to the final media output.
Feedback from participants in the technical run-throughs and pilot sessions were universally
positive with regards to the media delivery. The most significant change made was the addition of
a new section, presented as a "live video conference" from the Emergency Operations Centre in
Gapville, delivered by the Director of Emergency Management. This device allowed for further
dissemination of pertinent information. To better accommodate the time restraints of the session,
a second edit was made of the materials, and the final output was shortened by approximately 2
minutes. The paper-based materials in the participant binders reflected the changes made to the
scripts. A situation report in a government standard template, listing the major plot points was
provided for ease of reference.
44
'5'&&66&5
Process
The original PODS project process began with participants filling out various forms including the
consent form. These were immediately collected by the team members. A morning briefing
followed which introduces participants to the in-vivo experiment. After the briefing, participants
were taken to their respective rooms (or pods) by team members; in these rooms participants were
given their binders which contain envelopes with specific tasks and questionnaires. Participants
watched the simulation and were prompted visually to open different envelopes to discover
informational materials, tasks, and questionnaires for task one and task two. A 20 minute break
was provided to participants between the two tasks. Once task two was completed, participants
gathered in the original briefing room and were debriefed. Participants’ feedback was recorded
and considered along with team observer reports on the technical and process issues form (see
Annex P.4).
Major changes to the original process included:
• Collecting participant consent forms and other initial forms once participants have been
brought to their respective rooms.
• The addition of labeling to participant materials to ensure ethical standards of confidentiality
and ensure proper linking of information.
• Inter-pod communication methods were changed, so that participants are only able to
communicate through CB radios or the chat function for the first 20 minutes. Once the 20
minutes are over, participants are able to communicate via video conference for the
remainder of the task.
• Screen layout changes were made to the technical process, and made a difference
concerning how videos and information slides were to be presented.
The number of process and content problems decreased considerably from one session to the
next. Technical and process improvements were incorporated as opportunities arose. The end
result was an engaging, efficiently delivered simulation exercise that reflected a high professional
standard.
'5'&&66&5
45
4
4.1.1
Next steps: Running the In Vivo simulation
Scheduled sessions
A total of four additional sessions will be completed with senior officials. These sessions will be
held on April 6th 2011, April 13th 2011, May 5th 2011, and May 11th 2011.Two of these sessions
will have coordinative tasks, while the other two will have collaborative tasks. Also, two of the
sessions will have mixed pods, while the other two sessions will have homogenous pods. The mix
of pods and type of tasks has yet to be assigned to the specific dates.
4.1.2
4.1.2.1
Data collection methods and planned analysis
Component 1 – Qualitative interviews with decision makers
Data collection
Additional complementary interviews will be scheduled according to the participants’
availability. They will also be audio recorded, and will take place in a convenient location for the
participant (e.g. her /his office). These interviews will take place in the National Capital region.
Once the interviews are completed, the audio files will be secured at the Gap Santé facilities at
the University of Ottawa.
Data Transcription
Each interview will have to be transcribed completely in order to process the information with
qualitative analysis software that will classify, sort out and range the data previously imported, to
find trends and patterns that can be analysed.
Data Analysis
The first step to analyse the information transcribed, will be to detect elements related to
situational complexity (simple, complicated or complex). Each section will contain key pieces of
data. These pieces will become the themes to develop the analysis to validate the interorganizational problem solving model and to identify themes that could potentially inform the
framework.
In order to accomplish this, data will be categorized into few topics. Then each category will be
expanded depending on the level of specificity required to sustain each analysis. The reliability of
the proposed coding scheme will be assessed by comparing the 10% of the coding results of the
interviews, by two different researchers. The goal is to reach a consensus in the results, but if this
does not occur, iteration in the process for the elements under discussion is done, until the results
concur.
46
'5'&&66&5
4.1.2.2
Component 2 – In vivo simulation experiment
Data Collection
The data collected will include both self-report data collected from individual participants after
each task (See Annex G for the participant task questionnaires), and observational data collected
during the sessions via audio/video recording. The experiment will generate at least seven and a
half hours of audio visual materials per session, from approximately two and a half hours of pod
interactions at the different cameras located in various rooms (please refer to Section 3.9.4 for
details).
Data Cleaning and set up
All data will be set up for analysis by checking its quality and neatness. Data from the
participants’ questionnaires will be captured in a digital statistical database, while protecting the
anonymity of participants with the use of identifiers (please refer to Section 3.2.2 for details).
The data from the audio and video recordings will also be quality checked and then correlated
with the participant’s identifiers. Both the video and audio recordings will be processed by
applying picture filters to blur faces. Afterwards, both audio and video data will be transcribed.
This will require a considerable effort due to the length of the recordings and the complexity of
the participants’ interactions.
Data Analysis
The data will be coded and cleaned. Descriptive statistics will be run for each dependent variable
to evaluate data characteristics such as outliers, normality, linearity, variance homogeneity and
frequencies.
Due to the study’s experimental design, it is expected to apply variance and covariance analyses
such as ANOVA, ANCOVA, MANOVA, and MANCOVA. These analyses will allow testing the
main effects of the two independent variables (approach to multi-organizational decision-making
and multi-organizational environments), and their interactions. Based on the results from the
variance and covariance analysis, further comparison and contrast tests would be required to
allow for complementary interpretation of the main effects and its interactions. Also based on
correlation analysis, the relationship between some dependent variables can be further
understood.
4.1.3
Modelling communication and decision making functions
The goal for Task 4, given the results of the laboratory findings, is to develop a model of the
emergency management decision functions and the communications functions to support the
decision function. The model of these functions must describe limitations and variations based
upon situational factors described in the case studies. These situational factors must include
circumstances where there is the potential for failure (e.g., the failure of the communications
infrastructure during Hurricane Katrina). The functions must also be related to the SDM and ICS
frameworks as well as activities described as elements of capabilities within the National Incident
Management System (NIMS).
'5'&&66&5
47
References .....
Alison, L. & Crego, J. (2008). Policing critical incidents: Leadership and critical incident
management. Portland, OR: Willan Publishing.
Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of
Canada, and Social Sciences and Humanities Research Council of Canada. (2008). TriCouncil Policy Statement: Ethics Conduct for Research Involving Humans.
Crotty, M. (1998). The foundations of social research. London: SAGE.
Edwards, W., Miles Jr., R.F., & von Winterfeldt, D. (2007). Advances in Decision Analysis:
From Foundations to Applications. Cambridge, UK: Cambridge University Press.
Klein, G. A., Calderwood, R. & MacGregor, D. (1989). Critical Decision Method for eliciting
knowledge, IEEE Transactions on Systems, Man and Cybernetics, 19, 462 -472.
Lemyre, L., Boutette, P, Pinsent, C., Corneil, W., Johnson, C, Munoz, M., Lemieux, V., LalandeMarkon, M.P., Gibson, S., Riding, J., Lemus, C., Blust, S., and Dennie-Filion, E. (2009).
Research Using In Vivo Simulation of Meta-Organizational Shared Decision Making
(SDM): Task 1: Synthesis of Case Studies to form a SDM framework. Report for DRDC
Canada.
Lemyre, L., Boutette, P, Pinsent, C., Corneil, W., Johnson, C, Lalande-Markon, M.P., Gibson, S.,
Riding, J., Lemus, C., Blust, S., and Dennie-Filion, E. (2010). Research Using In Vivo
Simulation of Meta-Organizational Shared Decision Making (SDM): Task 2:
Development of an experimental plan for in vivo exercise and simulation. Report for DRDC
Canada.
Lindlof, T. R. and Taylor, B. C., (2002) Qualitative Communication Research Methods (second
Ed.), Sage Publications, Thousand Oaks, CA.
Nja, Ove; & Rake, E. (2009). An essay on research methodology: An alternative approach to
incident command research through participatory action research. International Journal
of Emergency Management, 6(1), 55-72.
Smith, J. A. (Ed.) (2008). Qualitative psychology: A practical guide to research methods.
Thousand Oaks: SAGE.
Smith, W., & Dowell, J. (2000). A case study of co-ordinative decision-making in disaster
management. Ergonomics, 43(8), 1153-1166.
48
'5'&&66&5
This page intentionally left blank.
'5'&&66&5
49
Annex A
Qualitative Interview Guide
Risk Perception and Risk Management: Multi Organizational Problem Solving
INTERVIEWER REFERENCE SHEET
Points to go over with respondent before the interview starts.
Context:
• These interviews are one component of a larger study examining multi-organizational problem
solving during emergencies and extreme events. Other main component is an experiment where
organizations go through an in vivo exercise and are required to problem solve with other types of
organizations.
• The information collected in the interviews will help in interpreting the findings from the experiment.
Content:
• Interviews are being conducted with senior level decision-makers who participated directly in
planning and/or responding during key events.
• Interview should take about one hour.
• We are interested in your perceptions of the challenges and opportunities that exist when making
decisions with multiple organizations involved.
• During the interview, we will:
o initially go through some background about how you were involved in [event]
o then get you to focus in on one specific challenging situation where you were actively
involved in problem solving and decision-making in a multi-organization context that was
particularly challenging, especially one that involved multiple sectors such as the
Canadian Forces, or NGOs like Red Cross or citizen associations, and multiple
jurisdictions such as Federal, Provincial, and Municipal
o ask you questions about how the group moved through the problem solving process
o get your thoughts on how some key concepts we are using in the experiment component
of the study
Questions before getting started?
50
'5'&&66&5
Participant overview
1. To get started, can you just briefly confirm for me the organization you were with during the event
and aspects like your title, main role and responsibilities, etc.? [we want to also understand their
training background both professional and experience-based – ask here if natural segue, or at the
end of the interview once rapport has been established]
Event overview and selection of situation
2. What are one or two situations planning for the event or during the event that stick out for you as
being particularly challenging – I need you to think of situations that focused on problem solving or
decision making, and that had a number of different types of organizations actively involved.
Briefly, what would these be? [who, what, where, when, why]
a. Which of these should we focus on for the remainder of the interview? [get any additional
details required]
Problem identification and definition
3. Thinking about the situation, what problem was the most challenging that had multiple
organizations involved? Who knew about the problem first? How did the other organizations
become involved? How was the problem defined once other organizations became involved?
What were the main areas of agreement/conflict among the players involved at this early stage?
Which organizations’ perspectives were dominant at this stage? Was there a lead organization or
person at this stage? How were they selected?
Solution generation
4. Once the problem had been defined, how did the group generate various potential solutions? What
were the solutions considered? What were the main areas of agreement/conflict among the
players when it came to solution generation? Which organizations’ perspectives were dominant at
this stage? Was there a lead organization or person at this stage? How were they selected?
Decision making and implementation
5. Once various potential solutions had been generated, how did the group decide which solutions to
implement? What were the main areas of agreement/conflict among the players when it came to
deciding on which solutions to implement? Which organizations’ perspectives were dominant at
this stage? Was there a lead organization or person at this stage? How were they selected?
Sharing resources and flexibility
6. Thinking back to this specific situation, what types of things were shared across organizations in
order to make the decision? To implement the solutions? – for example, information, resources,
leadership? [what, when, who]
Expectations and alternatives
7. Thinking back to just before this specific situation occurred, what would have been your
expectations with respect to how the situation would have played out with the organizations
involved? Considering how it rolled out, what aspects surprised you?
'5'&&66&5
51
8. Reflecting back on the situation, what should have been done differently? What should other
organizations have done differently? What would/could your organization have done differently?
Concepts
9. The experimental component focuses on the concepts of coordination, cooperation and
collaboration. When these terms are used, what examples or meanings would you assign to each
of them? Do you see them differently? How would you differentiate them? How do you see them
related to decision-making and problem-solving?
52
'5'&&66&5
Annex B
Interview Invitation Letter
Email request to organizations for recruitment
Dear Sir or Madam,
Your organization is invited by Dr Louise Lemyre from the Institute of Population Health at the University
of Ottawa to participate in an interview on Risk Perception and Risk Acceptability of Public Crisis,
Disasters and Emergencies, by sending this letter to some of your staff to contact us.
The goal is to conduct a series of interviews to better understand key actor’s and decision-maker’s
perception of disasters and public emergencies, the strategies employed to solve the problems faced during
complex crises, and their experience working with representatives from other organizations to solve public
emergencies. This part of the project is funded by the Public Safety Canada, National Defence Canada, the
Natural Sciences and Engineering Research Council of Canada, and the Social Sciences and Humanities
Research Council.
The participation of your organization is important to us. Managers or directors from your
organization, who have played a key role during a public crisis, such as the H1N1 plan, the SARS outbreak,
the 2003 Blackout, the 1998 Ice storm or the 1997 Red River Floods, are invited to participate in an
interview on risks, directed by a senior interviewer. An assistant may also be present to take notes, and a
voice recorder will be used to tape the session. The identity of the participants will not appear on any of the
written transcriptions. The analysis of the results will look at common themes and ideas among different
organizations. This will help to better understand the important factors influencing views on disasters and
public emergencies in Canadian communities. It will also serve to develop and evaluate best-practice
guidelines, to assist in managing psychosocial considerations of risk perception and risk acceptability.
The interview will last approximately 1 hour either at your offices or at the University depending on the
participants’ preference. Questions of the following type will be asked in relation to their perception of
crises, disasters or public emergencies: “From your perspective, what was the effect of multiple
organizations participating in joint problem solving? How did you experience uncertainty surrounding
these situations?” There are no right or wrong answers. We only want to know the participant’s experience.
Participation is anonymous. A code number will be attributed to participants. Confidentiality and
anonymity will be maintained by the research team. While global interview results of the study will be
published and discussed, within the research team and at research conferences, no identifiable individual
response will be shared. All of the data will be tabulated and maintained by the researchers in accordance
with research confidentiality standards. Participation is completely voluntary and anonymous. Participants
are free to withdraw at any time, refuse to participate, or refuse to answer certain questions. There is no
direct individual or organizational benefit from answering the questions. There is no expected harm or risk.
Please contact Dr Louise Lemyre’s assistant for participation, either by email at
gapsante@uottawa.ca or by phone at (613) 562-5800, extension 2321, as soon as possible.
Sincerely,
Professor Louise Lemyre, Ph.D., FRSC
(613)562 - 5800 (1196) ou (2321) Fax: (613)562 - 5350
'5'&&66&5
53
louise.lemyre@uottawa.ca, gapsante@uottawa.ca
Pavillon Desmarais Hall, 55 Laurier Ave E, pièce/room 3217
Ottawa (ON), Canada, K1N 6N5
www.gapsante.uottawa.ca
54
'5'&&66&5
Annex C
Interview Consent Form
Consent form and information sheet
Dear Sir or Madam,
You are invited by Dr Louise Lemyre from the Institute of Population Health at the University of Ottawa to
participate in an interview on Risk Perception and Risk Acceptability of Crises, Disasters and Emergencies.
The goal is to conduct a series of interviews to better understand the key decision maker`s perception of
disasters and public emergencies, the strategies employed by them to solve the problems faced during these
complex crises, and their experience working with representatives from other organizations to solve public
emergencies. This part of the project is funded by Public Safety Canada, National Defence Canada, the
Natural Sciences and Engineering Research Council of Canada, and the Social Sciences and Humanities
Research Council.
Your opinion is important to us. You will be asked to participate in an interview and to answer questions
directed by a trained interviewer. An assistant may also be present to take notes and a voice recorder will be
used to tape the session. All of the discussion will be transcribed and entered in a computer file for analysis
of content. The data will be conserved for a maximum of 10 years in a locked cabinet in the Gap-Santé
office at the University of Ottawa and will be shredded at the end of storage. Your identity will not appear
on any of the written transcriptions. The analysis of the results will look at common themes and ideas
among key decision makers from different organizations. This will help to better understand the important
factors influencing views on disasters and public emergencies in Canadian communities. It will also serve
to develop and evaluate best-practice guidelines, to assist health professionals and other key players to
effectively manage the short-term, mid-term and long-term psychosocial consequences of crises, disasters
and emergency events.
The interview will last approximately 1 hour. Questions of the following type will be asked in relation to
your perception of disasters and public emergencies: “From your perspective, what was the effect of
multiple organizations participating in joint problem solving? How did you experience uncertainty
surrounding these situations?” Your answers can be general and you do not have to reveal any information
that you do not want to. There are no right or wrong answers. We only want to know your experience.
Your participation is anonymous. Do not use your full name. Confidentiality and anonymity will be
maintained by the research team. While interview results of the study will be published and discussed
within the research team, and at research conferences, no one individual’s responses to the questions will
be shared. All of the data will be tabulated and maintained by the researchers in accordance with research
confidentiality standards.
Participation is completely voluntary and anonymous. You are free to withdraw at any time, refuse
to participate, or refuse to answer certain questions. There is no direct individual benefit from
answering the questions. There is no expected harm or risk except the possible negative feelings
'5'&&66&5
55
sometime associated with self-reflecting on crises, disasters and public emergencies. However, in the
unlikely event of distress or discomfort you may wish to contact the Confidential Help Line within
your locality. You may also contact Dr Louise Lemyre at the University of Ottawa at 613-562-5800,
extension 1196, or the University of Ottawa Research Ethics Board Officer at 613-562-5387
(ethics@uottawa.ca).
There are two copies of this form, one for you, one for the research team.
_____________________
Louise Lemyre, Ph.D.
University of Ottawa Professor
I agree to participate in this study about Risk Perception and Risk Acceptability of Crises, Disasters
and Public Emergencies conducted by Dr Louise Lemyre.
___________________________________
Participant, date
56
'5'&&66&5
Annex D
'5'&&66&5
Ethics Approval Letter
57
Annex E
Invitation Email for Recruitment
Email request to organizations for recruitment
Dear Sir or Madam,
I am inviting representatives from your organization to participate in a study at the
University of Ottawa. This study has been designed to help us better understand how
people in decision making roles perceive risks, assess problems and manage threats
during emergencies. This study involves having people from different organizations
participate in a session that is similar to a table-top exercise that focuses on a specific
emergency scenario. The title of the project is Risk Management and Governance:
Understanding Problem Solving and Decision Making, and it is being funded by the
Social Sciences and Humanities Research Council and Defence Research Development
Canada.
Who should participate?
We are hoping some representatives from your organization are able to participate in this
study. The participants for this study will consist of higher-level decision makers who
have had some previous experience in both strategic and operational decision-making in
managing and responding to threats. They should have been in a management-level
decision making role within your organization for at least one year. We will be inviting
participants from various organizations to participate in the same session.
What does participation involve?
The session will last approximately three hours and will resemble a table-top exercise
often used for emergency planning. The session will take place at the University of
Ottawa downtown campus during regular business hours. Participants will be asked to
interact with other session participants as they work through the exercises. As well, they
will be asked to complete various questionnaires on perceptions and opinions of tasks,
performance and interactions. All sessions will be video and audio recorded for data
collection purposes. All aspects of participation are voluntary. Participants will be asked
to read through and sign a consent form indicating informed consent prior to their
participation in the session.
What about confidentiality and anonymity?
Data collected from participants will remain confidential. The only people who will have
access to identifiable data will be members of the research team. Working files such as
transcripts and questionnaires will have identifiers (e.g., names, organization) removed
and replaced by an identification number. The questionnaires, audio and video tapes and
the transcriptions will be kept in a secure manner in a laboratory at the University of
58
'5'&&66&5
Ottawa to which only the researchers have access for a period of ten years. Participation
in the study will not be completely anonymous, as the participant’s organization will
know that he or she is participating, as will the fellow session participants. Beyond this,
the research team will protect the anonymity of participants by ensuring that participants’
names are not provided to any group outside of the research team directly involved with
this study.
What are the individual level risks and benefits associated with participating in this
study?
Participation is completely voluntary. Participants are free to withdraw at any time, refuse to
participate, or choose to not answer certain questions. We do not expect that there will be any
harm or potential risks with the exception of potentially negative feelings sometimes associated
with working directly with materials that focus on emergencies. The benefit of participation is
that the study will provide useful information for the emergency planning and response
community more broadly on how to potentially improve decision-making and management of
emergency events.
What are the next steps?
A member of my research team will be in contact with you either by phone or email over
the next few days to determine if there are any members of your organization that would
be interested in participating and how they can get in contact with the research team, and
to answer any additional questions you may have about the study. In the mean time, if
you have questions or would like further information, please contact me directly at (613)
562-5800 x 2321 or gapsante@uottawa.ca .
Sincerely,
Louise Lemyre, Ph.D., MSRC|FRSC
Professeure titulaire | Full Professor
École de psychologie, Faculté des sciences sociales | School of Psychology, Faculty of
Social Sciences
Chaire McLaughlin sur le risque psychosocial | The McLaughlin Chair on Psychosocial
Risk
Directrice de l'unité de recherche GAP-Santé | GAP-Santé research unit Director
Institut de santé des populations | Institute of Population Health
Université d'Ottawa | University of Ottawa
DMS 3215, 55 Laurier E
Ottawa, On, CANADA K1N 6N5
(+1) 613-562-5800 x1196 , assist. x2321
louise.lemyre@uottawa.ca ; gapsante@uottawa.ca
www.gapsante.uottawa.ca
'5'&&66&5
59
Annex F
Participant Consent Form
Information Sheet and Consent form
You have been invited to participate in a study entitled Risk Management and Governance:
Understanding Problem Solving and Decision Making: “Problem-solving and Organizational
Decision-making (PODS) Project” (the “Project”)This document provides information on the
study including the overall purpose of the study, what is involved in participating, possible risks
and benefits, how confidentiality and anonymity will be protected by the researchers, how data
will be conserved, and the voluntary nature of the study. As you read through this information
sheet, please note any questions that you have or anything that you would like to have clarified
by the researcher before you sign your acceptance to participate in the study.
Title of the study:
Risk Management and Governance: Understanding Problem
Solving and Decision Making: “Problem-solving and
Organizational Decision-making (PODS) Project”
Principal Investigator:
Louise Lemyre
Professor, School of Psychology
Faculty of Social Sciences
University of Ottawa
Tel: 613-562-5800 (ext 1196)
llemyre@uottawa.ca
Funders: This study is funded by the Social Sciences and Humanities Research Council, with
additional funding from Defense Research Development Canada.
Purpose of the Study: I understand that the purpose of the study is to better understand how
people in decision making roles perceive risks, assess problems and manage threats during
emergencies. This study involves having people from different organizations participate in a
session that is similar to a table-top exercise that focuses on a specific emergency scenario.
Participation: My participation in the study will consist of one session of approximately four
hours, located at the downtown campus of the University of Ottawa. During the session I will be
asked to participate in an exercise that resembles a table-top exercise often used for emergency
planning. I and the other participants assigned to my group for the session will be presented
with a scenario and asked to complete problem-solving tasks related to the scenario. I will be
asked to interact with other session participants as I work through the tasks. As well, I will be
asked to complete questionnaires that ask about my perceptions and opinions of the tasks,
60
'5'&&66&5
performance and interactions. The session will be video and audio recorded for data collection
purposes, and later transcribed in an electronic file.
Risks: I understand that this research is devoid of physical and psychological risks other than the
possible mild negative feelings that could be associated with working with a scenario that
focuses on an emergency situation. The research team has selected the materials and
developed the scenario in a manner that attempts to minimize this risk by not including graphic
descriptions or negative images.
Benefits: My participation in this study will help improve understanding and provide useful
information for the emergency planning and response community on how to potentially
improve decision-making and management of emergency events.
Confidentiality and Anonymity: I have received assurance from the researcher that the
information I will share will remain strictly confidential. I understand that the contents will be
used only for research purposes and that my confidentiality will be protected: only grouped
data will be reported. My participation in the study will not be completely anonymous, as the
fellow session participants will know that I am participating. Beyond this, the research team will
protect my anonymity by ensuring that my name is not provided to any group outside of the
research team directly involved with this study.
Conservation of Data: The questionnaires, audio and video tapes and the transcriptions will be
kept in a secure manner in a laboratory at the University of Ottawa to which only the
researchers have access for a period of ten years.
Voluntary Participation: I am under no obligation to participate and if I choose to participate, I
may withdraw from the study at any time. I may also refuse to answer any questions, without
suffering any negative consequences. If I choose to withdraw, all data gathered until the time of
withdrawal will be destroyed and will not be used.
Acceptance: Participation is completely voluntary. You are free to withdraw at any time, refuse
to participate, or refuse to answer certain questions. There is no direct individual benefit from
answering the questions. There is no expected harm or risk except the possible negative feelings
sometime associated with self-reflecting on public emergencies. However, in the unlikely event
of distress or discomfort you may wish to contact the Confidential Help Line within your locality.
You may also contact Dr Louise Lemyre at the University of Ottawa at 613-562-5800, extension
1196, or the University of Ottawa Research Ethics Board Officer at 613-562-5387, University of
Ottawa, Tabaret Hall, 550 Cumberland Street, Room 159, Ottawa, ON (ethics@uottawa.ca).I
agree to participate in this study Risk Management and Governance: Understanding Problem
Solving and Decision Making: “Problem-solving and Organizational Decision-making (PODS)
Project” conducted by Dr Louise Lemyre.
PARTICIPANT:
'5'&&66&5
DATE:
61
Louise Lemyre, Ph.D.University of Ottawa Professor
62
'5'&&66&5
Annex G
G.1
Task Questionnaires
Open Pod Task #1 Questionnaire
Do not complete this questionnaire until instructed to do so
Individual Participation and Perspectives - Task Questionnaire
Please reflect on your own participation and thoughts during the task that you just completed.
Read each of the following statements and then rate your level of agreement on a scale of 0 to 4
(0 – strongly disagree; 1 – disagree; 2 – neutral; 3 – agree; 4 – strongly agree).
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q1
I am satisfied with the
problem solving processes I
used during the task.
ż
ż
ż
ż
ż
Q2
I am satisfied with the
opportunities I had to provide
input.
ż
ż
ż
ż
ż
Q3
I am satisfied with the overall
quality of the outcome from
the task.
ż
ż
ż
ż
ż
Q4
I am frustrated working with
the people in my pod group.
ż
ż
ż
ż
ż
Q5
I am frustrated working with
the people in the other pods.
ż
ż
ż
ż
ż
Q6
I participated actively in the
decision making process.
ż
ż
ż
ż
ż
Q7
I participated in a leadership
role within my own pod.
ż
ż
ż
ż
ż
Q8
I participated in a leadership
role across the other pods.
ż
ż
ż
ż
ż
'5'&&66&5
63
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q9
I facilitated discussion within
my own pod.
ż
ż
ż
ż
ż
Q10
I facilitated discussion
between pods.
ż
ż
ż
ż
ż
Q11
I recorded information on
behalf of my own pod.
ż
ż
ż
ż
ż
Q12
I recorded information on
behalf of all pods
ż
ż
ż
ż
ż
Q13
I was frustrated by
differences of opinion within
my pod during the task.
ż
ż
ż
ż
ż
Q14
I was frustrated by
differences of opinion
between the other pods
during the task.
ż
ż
ż
ż
ż
Q15
I trust the people in my pod.
ż
ż
ż
ż
ż
Q16
I trust the people in the other
pods.
ż
ż
ż
ż
ż
Q17
I felt a sense of belonging
within my pod.
ż
ż
ż
ż
ż
Q18
I felt a sense of belonging
with the other pods.
ż
ż
ż
ż
ż
Q19
I agree with the decisions and
outcomes from the task.
ż
ż
ż
ż
ż
Q20
Prior to today’s session I
would describe my
relationship with at least one
of the people in my pod as a
friendship.
ż
ż
ż
ż
ż
64
'5'&&66&5
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q22
Prior to today’s session, I
have worked with at least one
of the people in my pod.
ż
ż
ż
ż
ż
Q24
There was enough time
allotted to solve the problems
during the task.
ż
ż
ż
ż
ż
Q25
The information and
instructions given by the
session facilitator were clear.
ż
ż
ż
ż
ż
Q26
The task was difficult to
complete.
ż
ż
ż
ż
ż
'5'&&66&5
65
Within POD Participation and Perspectives - Task Questionnaire
Please reflect on the participation of the various people in your pod during the task that you
just completed. Read each of the following statements and then rate your level of agreement on a
scale of 0 to 4 (0 – strongly disagree; 1 – disagree; 2 – neutral; 3 – agree; 4 – strongly agree).
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q27
The outcome of the task
reflects the input from all the
people in my pod.
ż
ż
ż
ż
ż
Q28
The differences of opinion
within my pod helped us
reach better decisions during
the task.
ż
ż
ż
ż
ż
Q29
The people in my pod
actively participated in the
problem solving process.
ż
ż
ż
ż
ż
Q30
The people in my pod
communicated effectively
with one another.
ż
ż
ż
ż
ż
Q31
The people in my pod were
engaged in the decision
making process.
ż
ż
ż
ż
ż
Q32
The people in my pod
generated various alternative
ideas.
ż
ż
ż
ż
ż
Q33
A clear leader emerged
within my pod.
ż
ż
ż
ż
ż
Q34
There was more than one
leader within my pod.
ż
ż
ż
ż
ż
Q35
Personality styles within my
pod got in the way of
completing the task.
ż
ż
ż
ż
ż
66
'5'&&66&5
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q36
The people within my pod
had frequent differences of
opinion.
ż
ż
ż
ż
ż
Q37
The people in my pod were
motivated to complete the
task.
ż
ż
ż
ż
ż
Q38
The people in my pod were
able to stay focused on the
task.
ż
ż
ż
ż
ż
'5'&&66&5
67
Between PODS Participation and Perspectives - Task Questionnaire
Please reflect on the participation of people from all the pods during the task that you just
completed. Read each of the following statements and then rate your level of agreement on a
scale of 0 to 4 (0 – strongly disagree; 1 – disagree; 2 – neutral; 3 – agree; 4 – strongly agree).
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q39
The differences of opinion
across the pods helped us to
reach a better decision during
the task.
ż
ż
ż
ż
ż
Q40
People from the other pods
actively participated with
people from my pod in the
problem solving process.
ż
ż
ż
ż
ż
Q41
The pods communicated
effectively with one another.
ż
ż
ż
ż
ż
Q42
People from the other pods
were engaged in the decision
making process.
ż
ż
ż
ż
ż
Q43
People from the other pods
generated various alternative
ideas.
ż
ż
ż
ż
ż
Q44
A clear leader emerged
between the pods.
ż
ż
ż
ż
ż
Q45
Personality styles between
people in the different pods
got in the way of completing
the task.
ż
ż
ż
ż
ż
Q46
People in the other pods had
frequent differences of
opinion.
ż
ż
ż
ż
ż
Q47
The people from the other
pods were motivated to
complete the task.
ż
ż
ż
ż
ż
68
'5'&&66&5
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q48
The pods were able to stay
focused on the task.
ż
ż
ż
ż
ż
Q49
Decisions made between my
pod and the other pods were
consensus-based.
ż
ż
ż
ż
ż
Q21
Prior to today’s session I
would describe my
relationship with at least one
of the people in the other
pods as a friendship.
ż
ż
ż
ż
ż
Prior to today’s session, I
have worked with at least one
of the people from the other
pods.
ż
ż
ż
ż
ż
Q23
'5'&&66&5
69
G.2
Open Pod Task #2 Questionnaire
Do not complete this questionnaire until instructed to do so
Individual Participation and Perspectives - Task Questionnaire
Please reflect on your own participation and thoughts during the task that you just completed.
Read each of the following statements and then rate your level of agreement on a scale of 0 to 4
(0 – strongly disagree; 1 – disagree; 2 – neutral; 3 – agree; 4 – strongly agree).
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q1
I am satisfied with the
problem solving processes I
used during the task.
ż
ż
ż
ż
ż
Q2
I am satisfied with the
opportunities I had to provide
input.
ż
ż
ż
ż
ż
Q3
I am satisfied with the overall
quality of the outcome from
the task.
ż
ż
ż
ż
ż
Q4
I am frustrated working with
the people in my pod group.
ż
ż
ż
ż
ż
Q5
I am frustrated working with
the people in the other pods.
ż
ż
ż
ż
ż
Q6
I participated actively in the
decision making process.
ż
ż
ż
ż
ż
Q7
I participated in a leadership
role within my own pod.
ż
ż
ż
ż
ż
Q8
I participated in a leadership
role across the other pods.
ż
ż
ż
ż
ż
Q9
I facilitated discussion within
my own pod.
ż
ż
ż
ż
ż
70
'5'&&66&5
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q10
I facilitated discussion
between pods.
ż
ż
ż
ż
ż
Q11
I recorded information on
behalf of my own pod.
ż
ż
ż
ż
ż
Q12
I recorded information on
behalf of all pods
ż
ż
ż
ż
ż
Q13
I was frustrated by
differences of opinion within
my pod during the task.
ż
ż
ż
ż
ż
Q14
I was frustrated by
differences of opinion
between the other pods
during the task.
ż
ż
ż
ż
ż
Q15
I trust the people in my pod.
ż
ż
ż
ż
ż
Q16
I trust the people in the other
pods.
ż
ż
ż
ż
ż
Q17
I felt a sense of belonging
within my pod.
ż
ż
ż
ż
ż
Q18
I felt a sense of belonging
with the other pods.
ż
ż
ż
ż
ż
Q19
I agree with the decisions and
outcomes from the task.
ż
ż
ż
ż
ż
Q24
There was enough time
allotted to solve the problems
during the task.
ż
ż
ż
ż
ż
Q25
The information and
instructions given by the
session facilitator were clear.
ż
ż
ż
ż
ż
Q26
The task was difficult to
complete.
ż
ż
ż
ż
ż
'5'&&66&5
71
Within POD Participation and Perspectives - Task Questionnaire
Please reflect on the participation of the various people in your pod during the task that you
just completed. Read each of the following statements and then rate your level of agreement on a
scale of 0 to 4 (0 – strongly disagree; 1 – disagree; 2 – neutral; 3 – agree; 4 – strongly agree).
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q27
The outcome of the task
reflects the input from all the
people in my pod.
ż
ż
ż
ż
ż
Q28
The differences of opinion
within my pod helped us
reach better decisions during
the task.
ż
ż
ż
ż
ż
Q29
The people in my pod
actively participated in the
problem solving process.
ż
ż
ż
ż
ż
Q30
The people in my pod
communicated effectively
with one another.
ż
ż
ż
ż
ż
Q31
The people in my pod were
engaged in the decision
making process.
ż
ż
ż
ż
ż
Q32
The people in my pod
generated various alternative
ideas.
ż
ż
ż
ż
ż
Q33
A clear leader emerged
within my pod.
ż
ż
ż
ż
ż
Q34
There was more than one
leader within my pod.
ż
ż
ż
ż
ż
Q35
Personality styles within my
pod got in the way of
completing the task.
ż
ż
ż
ż
ż
Q36
The people within my pod
had frequent differences of
opinion.
ż
ż
ż
ż
ż
72
'5'&&66&5
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q37
The people in my pod were
motivated to complete the
task.
ż
ż
ż
ż
ż
Q38
The people in my pod were
able to stay focused on the
task.
ż
ż
ż
ż
ż
'5'&&66&5
73
Between PODS Participation and Perspectives - Task Questionnaire
Please reflect on the participation of people from all the pods during the task that you just
completed. Read each of the following statements and then rate your level of agreement on a
scale of 0 to 4 (0 – strongly disagree; 1 – disagree; 2 – neutral; 3 – agree; 4 – strongly agree).
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q39
The differences of opinion
across the pods helped us to
reach a better decision during
the task.
ż
ż
ż
ż
ż
Q40
People from the other pods
actively participated with
people from my pod in the
problem solving process.
ż
ż
ż
ż
ż
Q41
The pods communicated
effectively with one another.
ż
ż
ż
ż
ż
Q42
People from the other pods
were engaged in the decision
making process.
ż
ż
ż
ż
ż
Q43
People from the other pods
generated various alternative
ideas.
ż
ż
ż
ż
ż
Q44
A clear leader emerged
between the pods.
ż
ż
ż
ż
ż
Q45
Personality styles between
people in the different pods
got in the way of completing
the task.
ż
ż
ż
ż
ż
Q46
People in the other pods had
frequent differences of
opinion.
ż
ż
ż
ż
ż
Q47
The people from the other
pods were motivated to
complete the task.
ż
ż
ż
ż
ż
74
'5'&&66&5
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q48
The pods were able to stay
focused on the task.
ż
ż
ż
ż
ż
Q49
Decisions made between my
pod and the other pods were
consensus-based.
ż
ż
ż
ż
ż
'5'&&66&5
75
G.3
Closed Pod Task #1 Questionnaire
Do not complete this questionnaire until instructed to do so
Individual Participation and Perspectives - Task Questionnaire
Please reflect on your own participation and thoughts during the task that you just completed.
Read each of the following statements and then rate your level of agreement on a scale of 0 to 4
(0 – strongly disagree; 1 – disagree; 2 – neutral; 3 – agree; 4 – strongly agree).
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q1
I am satisfied with the
problem solving processes I
used during the task.
ż
ż
ż
ż
ż
Q2
I am satisfied with the
opportunities I had to provide
input.
ż
ż
ż
ż
ż
Q3
I am satisfied with the overall
quality of the outcome from
the task.
ż
ż
ż
ż
ż
Q4
I am frustrated working with
the people in my pod group.
ż
ż
ż
ż
ż
Q6
I participated actively in the
decision making process.
ż
ż
ż
ż
ż
Q7
I participated in a leadership
role within my own pod.
ż
ż
ż
ż
ż
Q9
I facilitated discussion within
my own pod.
ż
ż
ż
ż
ż
Q11
I recorded information on
behalf of my own pod.
ż
ż
ż
ż
ż
Q13
I was frustrated by
differences of opinion within
ż
ż
ż
ż
ż
76
'5'&&66&5
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
my pod during the task.
Q15
I trust the people in my pod.
ż
ż
ż
ż
ż
Q17
I felt a sense of belonging
within my pod.
ż
ż
ż
ż
ż
Q19
I agree with the decisions and
outcomes from the task.
ż
ż
ż
ż
ż
Q20
Prior to today’s session I
would describe my
relationship with at least one
of the people in my pod as a
friendship.
ż
ż
ż
ż
ż
Q22
Prior to today’s session, I
have worked with at least one
of the people in my pod.
ż
ż
ż
ż
ż
Q24
There was enough time
allotted to solve the problems
during the task.
ż
ż
ż
ż
ż
Q25
The information and
instructions given by the
session facilitator were clear.
ż
ż
ż
ż
ż
Q26
The task was difficult to
complete.
ż
ż
ż
ż
ż
'5'&&66&5
77
Within POD Participation and Perspectives - Task Questionnaire
Please reflect on the participation of the various people in your pod during the task that you
just completed. Read each of the following statements and then rate your level of agreement on a
scale of 0 to 4 (0 – strongly disagree; 1 – disagree; 2 – neutral; 3 – agree; 4 – strongly agree).
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q27
The outcome of the task
reflects the input from all the
people in my pod.
ż
ż
ż
ż
ż
Q28
The differences of opinion
within my pod helped us
reach better decisions during
the task.
ż
ż
ż
ż
ż
Q29
The people in my pod
actively participated in the
problem solving process.
ż
ż
ż
ż
ż
Q30
The people in my pod
communicated effectively
with one another.
ż
ż
ż
ż
ż
Q31
The people in my pod were
engaged in the decision
making process.
ż
ż
ż
ż
ż
Q32
The people in my pod
generated various alternative
ideas.
ż
ż
ż
ż
ż
Q33
A clear leader emerged
within my pod.
ż
ż
ż
ż
ż
Q34
There was more than one
leader within my pod.
ż
ż
ż
ż
ż
Q35
Personality styles within my
pod got in the way of
completing the task.
ż
ż
ż
ż
ż
Q36
The people within my pod
had frequent differences of
opinion.
ż
ż
ż
ż
ż
78
'5'&&66&5
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q37
The people in my pod were
motivated to complete the
task.
ż
ż
ż
ż
ż
Q38
The people in my pod were
able to stay focused on the
task.
ż
ż
ż
ż
ż
'5'&&66&5
79
G.4
Closed Pod Task #2 Questionnaire
Do not complete this questionnaire until instructed to do so
Individual Participation and Perspectives - Task Questionnaire
Please reflect on your own participation and thoughts during the task that you just completed.
Read each of the following statements and then rate your level of agreement on a scale of 0 to 4
(0 – strongly disagree; 1 – disagree; 2 – neutral; 3 – agree; 4 – strongly agree).
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q1
I am satisfied with the
problem solving processes I
used during the task.
ż
ż
ż
ż
ż
Q2
I am satisfied with the
opportunities I had to provide
input.
ż
ż
ż
ż
ż
Q3
I am satisfied with the overall
quality of the outcome from
the task.
ż
ż
ż
ż
ż
Q4
I am frustrated working with
the people in my pod group.
ż
ż
ż
ż
ż
Q6
I participated actively in the
decision making process.
ż
ż
ż
ż
ż
Q7
I participated in a leadership
role within my own pod.
ż
ż
ż
ż
ż
Q9
I facilitated discussion within
my own pod.
ż
ż
ż
ż
ż
Q11
I recorded information on
behalf of my own pod.
ż
ż
ż
ż
ż
Q13
I was frustrated by
differences of opinion within
my pod during the task.
ż
ż
ż
ż
ż
80
'5'&&66&5
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q15
I trust the people in my pod.
ż
ż
ż
ż
ż
Q17
I felt a sense of belonging
within my pod.
ż
ż
ż
ż
ż
Q19
I agree with the decisions and
outcomes from the task.
ż
ż
ż
ż
ż
Q24
There was enough time
allotted to solve the problems
during the task.
ż
ż
ż
ż
ż
Q25
The information and
instructions given by the
session facilitator were clear.
ż
ż
ż
ż
ż
Q26
The task was difficult to
complete.
ż
ż
ż
ż
ż
'5'&&66&5
81
Within POD Participation and Perspectives - Task Questionnaire
Please reflect on the participation of the various people in your pod during the task that you
just completed. Read each of the following statements and then rate your level of agreement on a
scale of 0 to 4 (0 – strongly disagree; 1 – disagree; 2 – neutral; 3 – agree; 4 – strongly agree).
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q27
The outcome of the task
reflects the input from all the
people in my pod.
ż
ż
ż
ż
ż
Q28
The differences of opinion
within my pod helped us
reach better decisions during
the task.
ż
ż
ż
ż
ż
Q29
The people in my pod
actively participated in the
problem solving process.
ż
ż
ż
ż
ż
Q30
The people in my pod
communicated effectively
with one another.
ż
ż
ż
ż
ż
Q31
The people in my pod were
engaged in the decision
making process.
ż
ż
ż
ż
ż
Q32
The people in my pod
generated various alternative
ideas.
ż
ż
ż
ż
ż
Q33
A clear leader emerged
within my pod.
ż
ż
ż
ż
ż
Q34
There was more than one
leader within my pod.
ż
ż
ż
ż
ż
Q35
Personality styles within my
pod got in the way of
completing the task.
ż
ż
ż
ż
ż
Q36
The people within my pod
had frequent differences of
opinion.
ż
ż
ż
ż
ż
82
'5'&&66&5
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q37
The people in my pod were
motivated to complete the
task.
ż
ż
ż
ż
ż
Q38
The people in my pod were
able to stay focused on the
task.
ż
ż
ż
ż
ż
'5'&&66&5
83
Annex H
1.
2.
3.
4.
5.
Background Information Questionnaire
:KDWLV\RXUFXUUHQWMREWLWOH"BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
+RZORQJKDYH\RXUZRUNHGLQWKLVSRVLWLRQ"BBBBBBPRQWKVBBBBBBB\HDUV
+RZORQJKDYH\RXZRUNHGIRUWKLVRUJDQL]DWLRQ"BBBBBBBPRQWKVBBBBBBB\HDUV
+RZORQJKDYH\RXZRUNHGLQHPHUJHQF\PDQDJHPHQW"BBBBBBPRQWKVBBBBBB
\HDUV
:KDWDUH\RXUPDLQUROHVDQGUHVSRQVLELOLWLHVZLWKUHVSHFWWR
a. 3ODQQLQJIRUHPHUJHQFLHV
b. 5HVSRQGLQJWRHPHUJHQFLHV
6. :KDWLV\RXUDFDGHPLFEDFNJURXQG"
a. &ROOHJHGLSORPDV
b. 8QLYHUVLW\GHJUHHV
7. :KDWLV\RXUFXUUHQWDJH"BBBBBBB\HDUV
8. *HQGHU
9. :KLFKODQJXDJHVDUH\RXIOXHQWHQRXJKLQWRZRUNLQ"&KHFNDOOWKDWDSSO\
† French
84
'5'&&66&5
† English
† Other
If Other, please list:
'5'&&66&5
85
Annex I
Subject:
Confidentiality / Intellectual Property
Agreement
Agreement for the University of Ottawa project entitled “In Vivo Simulation
of Meta-Organizational Shared Decision Making” (the “Project”)
In order to protect confidentiality of materials created and data collected over the course of the
“In Vivo Simulation of Meta-Organizational Shared Decision Making” project, we are kindly
asking reviewers of materials to agree to the confidentiality and intellectual property clauses
described below.
Confidentiality:
1.1
“Confidential Information” means all data and information relating to the research
project.
1.2
I understand and agree that, as part of my participation, I may have access to
Confidential Information belonging to the University of Ottawa or other participants
including, without limitation, the results of this project. I will regard such Confidential
Information as strictly confidential and agree not to reveal any such Confidential
Information to any other person, firm, corporation, company or entity unless specifically
authorized to do so in writing by Louise Lemyre, the Principal Investigator of the
Project. This obligation of secrecy will continue after my participation in the Project has
ended.
Intellectual Property:
2.1
“Intellectual Property” includes, but is not limited to: technical information, know-how,
copyrights, patents, models, patterns, drawings, specification, prototypes, and inventions.
2.2
I agree to assign ownership of all Intellectual Property I contribute to create, develop or
produce as part of my participation in the Project to the University of Ottawa. I agree to
execute such additional conveyances and other documents conveying title or copyright to
such Intellectual Property to the University of Ottawa as the University of Ottawa may
require. Should any Intellectual Property assigned to the University of Ottawa be in the
form of copyrights, as an author of copyrighted Intellectual Property, I hereby
permanently waive my moral rights, as defined in the Copyright Act, in respect such
Intellectual Property.
Signed:
______________________________
____________________
(Witness)
Name:_________________________
Date:____________________
86
'5'&&66&5
Annex J
Participant ID & Information Linking
Form
Participant
Name:
Participant
Label:
Session ż1 University of Ottawa
Type (check Students
one):
ż2 Professional Students
Session #:
ż3 Senior Officials
Pod #: ż1 One
ż2 Two
ż3 Three
Participant ż1 One
#:
ż2 Two
ż3 Three
ż4 Four
Participant ż1 Military
Type:
ż2 ICS Oriented
Participant
Organization:
ż3 Non-ICS Oriented
Pod Status: ż1 Connected
ż2 Closed
# ż1 One
Participants
in Pod: ż2 Two
ż3 Three
'5'&&66&5
87
Annex K
K.1
Task Worksheets
Open Pod Collaboration Task Worksheet #1
INSTRUCTIONS
• First 20 minutes:
o Work with the members of your pod on the tasks outlined in this worksheet.
o You can use the 2-way radios or chat function to communicate with the other
pods.
o Please record your responses directly on the worksheet – these will be collected
at the end of the session.
• Next 35 minutes:
o The video conference line will be open with the other pods.
o Continue to work on the tasks outlined in the worksheet.
o Continue to record your responses directly on the worksheet.
• Final 15 minutes:
o When directed, complete the questionnaire that was included in the envelope.
o Your completed questionnaire will be collected at the end of the session.
BACKGROUND INFORMATION FOR TASK
• Over the past 24 hours, there have been media reports that martial law and curfews are
going to be applied. Rumours are circulating that special legislation or the Emergencies
Act will be declared, giving military greater control of the city and any movement in or
out of it.
• While there is no truth to any of these rumours, there are armed and protected soldiers
who are guarding various sites and moving throughout the city to detect additional
sources of contamination; sightings of these soldiers are adding fuel to rumours.
• As a result, nervous citizens are evacuating prematurely, ignoring instructions to shelter
in place.
• There are increasing levels of criticism and concern from both citizens and local political
representatives that these communication issues have not been adequately addressed.
• Criticisms include conflicting instructions to the public, messages that are vague and
unclear, and major gaps in information.
• In order to address these criticisms, please work both within your own pod and with the
other two pods for the following tasks outlined in the worksheet.
88
'5'&&66&5
1) Come to a consensus across all organizations within all the pods on the two priority
messages or pieces of information that the group feels it is most important to relay to
the general public at this point. Record these messages in the form below.
MESSAGE #1:
MESSAGE #2:
'5'&&66&5
89
2) In consultation with the other organizations in all the pods, identify two potential
options for approaches that could be used by the group to effectively deliver each of
the two priority messages to the public. In developing approaches, consider who will
deliver the message, when it can be delivered, what venue or method would be
effective, and considerations to take in account if the option was to be selected.
MESSAGE #1 – OPTION #1
Main messenger:
Timing of message:
Venue/method:
Considerations:
MESSAGE #1 – OPTION #2
Main messenger:
Timing of message:
Venue/method:
Considerations:
90
'5'&&66&5
MESSAGE #2 – OPTION #1
Main messenger:
Timing of message:
Venue/method:
Considerations:
MESSAGE #2 – OPTION #2
Main messenger:
Timing of message:
Venue/method:
Considerations:
'5'&&66&5
91
3) As a group, make decisions on which option for each message is most appropriate.
MESSAGE #1 DECISION
Decision:
Rationale for decision:
MESSAGE #2 DECISION
Decision:
Rationale for decision:
92
'5'&&66&5
K.2
Open Pod Collaboration Task Worksheet #2
INSTRUCTIONS
• First 20 minutes:
o Work with the members of your pod on the tasks outlined in this worksheet.
o You can use the 2-way radios or chat function to communicate with the other
pods.
o Please record your responses directly on the worksheet – these will be collected
at the end of the session.
• Next 35 minutes:
o The video conference line will be open with the other pods.
o Continue to work on the tasks outlined in the worksheet.
o Continue to record your responses directly on the worksheet.
• Final 15 minutes:
o When directed, complete the questionnaire that was included in the envelope.
o Your completed questionnaire will be collected at the end of the session.
BACKGROUND INFORMATION FOR TASK
• Responders travelling from other jurisdictions, contractors, volunteers as well as the
various unions for the professional responders are all questioning whether or not they
are being adequately protected and what standards for personal protective equipment
should be imposed.
• Differing protocols and equipment are being used in the same sites, while multiple and
shifting hot zones are complicating matters further.
• Another hotly debated issue is the access to Prussian Blue for responders and their
families.
• Some members of the public feel that there is preferential treatment for these groups,
while those with access are raising questions about the side effects and long term
consequences.
• Occupational health and safety for traditional and non-traditional responders has
become a major issue in the context of the scenario, both in the short term and in the
longer term.
'5'&&66&5
93
• In order to address these concerns, please work both within your own pod and with the
other two pods for the following tasks outlined in the worksheet.
94
'5'&&66&5
1) Come to a consensus across all organizations within all the pods on the two most
significant responder health and safety issues that you feel are most important to address.
Record these issues in the form below.
HEALTH & SAFETY ISSUE #1:
HEALTH & SAFETY ISSUE #2:
'5'&&66&5
95
2) In consultation with the other organizations in all pods, identify two potential options for
approaches that could be used by the group to effectively address the two health & safety
issues. In developing approaches, consider who will deliver the interventions, when it can
be delivered, what method would be effective, and considerations to take in account if
the option was to be selected. Describe these approaches in the forms below.
HEALTH & SAFETY ISSUE #1 – OPTION #1
Who will deliver the interventions:
Timing:
Method:
Considerations:
HEALTH & SAFETY ISSUE #1 – OPTION #2
Who will deliver the interventions:
Timing:
Method:
Considerations:
96
'5'&&66&5
HEALTH & SAFETY ISSUE #2 – OPTION #1
Who will deliver the interventions:
Timing:
Method:
Considerations:
HEALTH & SAFTEY ISSUE #2 – OPTION #2
Who will deliver the interventions:
Timing:
Method:
Considerations:
'5'&&66&5
97
3) As a group, make decisions on which option for each health and safety issue is most
appropriate. Record the decisions and rationale in the grid below.
HEALTH & SAFETY ISSUE #1:
Decision on which option to act on:
Rationale for the decision:
HEALTH & SAFETY ISSUE #2:
Decision on which option to act on:
Rationale for the decision:
98
'5'&&66&5
K.3
Open Pod Coordination Task Worksheet #1
INSTRUCTIONS
• First 20 minutes:
o Work with the members of your pod on the tasks outlined in this worksheet.
o You can use the 2-way radios or chat function to communicate with the other
pods.
o Please record your responses directly on the worksheet – these will be collected
at the end of the session.
• Next 35 minutes:
o The video conference line will be open with the other pods.
o Continue to work on the tasks outlined in the worksheet.
o Continue to record your responses directly on the worksheet.
• Final 15 minutes:
o When directed, complete the questionnaire that was included in the envelope.
o Your completed questionnaire will be collected at the end of the session.
BACKGROUND INFORMATION FOR TASK
• Over the past 24 hours, there have been media reports that martial law and curfews are
going to be applied. Rumours are circulating that special legislation or the Emergencies
Act will be declared, giving military greater control of the city and any movement in or
out of it.
• While there is no truth to any of these rumours, there are armed and protected soldiers
who are guarding various sites and moving throughout the city to detect additional
sources of contamination; sightings of these soldiers are adding fuel to rumours.
• As a result, nervous citizens are evacuating prematurely, ignoring instructions to shelter
in place.
• There are increasing levels of criticism and concern from both citizens and local political
representatives that these communication issues have not been adequately addressed.
• Criticisms include conflicting instructions to the public, messages that are vague and
unclear, and major gaps in information.
• In order to address these criticisms, please work both within your own pod and with the
other two pods for the following tasks outlined in the worksheet.
'5'&&66&5
99
1) Determine the two main messages or pieces of information that you feel that it is most
important for you to relay to the general public at this point. Record these messages in
the form below. Then consult with others in your pod and in other pods to understand
their priority messages. Use the form below to keep track of their messages.
YOUR PRIORITY MESSAGES:
MESSAGE #1:
MESSAGE #2:
100
'5'&&66&5
PRIORITY MESSAGES FROM OTHERS IN YOUR POD
Organization:
Message #1
Message #2
Organization:
Message #1
Message #2
'5'&&66&5
101
PRIORITY MESSAGES FROM OTHERS IN OTHER PODS
Organization:
Message #1
Message #2
Organization:
Message #1
Message #2
Organization:
Message #1
Message #2
Organization:
Message #1
Message #2
Organization:
Message #1
Message #2
Organization:
Message #1
Message #2
102
'5'&&66&5
2) In consultation with the other members in all the pods, determine where there are
overlaps in messages, potential gaps or contradictions with respect to YOUR messages.
Record the overlaps and gaps in the form below.
YOUR MESSAGE #1:
Identified overlaps, gaps, and contradictions with other members` messages:
YOUR MESSAGE #2:
Identified overlaps, gaps, and contradictions with other members` messages:
'5'&&66&5
103
3) Work with the others in your pod to determine options for combining or grouping
messages with other members, and then decide on the most appropriate grouping and
methods of delivery for YOUR messages that need to be delivered to the public taking into
account options identified. Record the options and decisions in the grid below.
YOUR MESSAGE #1:
Potential options for grouping/combining with other messages:
Decision on grouping or not grouping with other messages
Rationale for decision on grouping or not grouping and any changes required to
message if grouping:
Decision on how message will be delivered (who, when, venue/method)
YOUR MESSAGE #2:
Potential options for grouping/combining with other messages:
Decision on grouping or not grouping with other messages
Rationale for decision on grouping or not grouping and any changes required to
message if grouping:
Decision on how message will be delivered (who, when, venue/method)
104
'5'&&66&5
K.4
Open Pod Coordination Task Worksheet #2
INSTRUCTIONS
• First 20 minutes:
o Work with the members of your pod on the tasks outlined in this worksheet.
o You can use the 2-way radios or chat function to communicate with the other
pods.
o Please record your responses directly on the worksheet – these will be collected
at the end of the session.
• Next 35 minutes:
o The video conference line will be open with the other pods.
o Continue to work on the tasks outlined in the worksheet.
o Continue to record your responses directly on the worksheet.
• Final 15 minutes:
o When directed, complete the questionnaire that was included in the envelope.
o Your completed questionnaire will be collected at the end of the session.
BACKGROUND INFORMATION FOR TASK
• Responders travelling from other jurisdictions, contractors, volunteers as well as the
various unions for the professional responders are all questioning whether or not they
are being adequately protected and what standards for personal protective equipment
should be imposed.
• Differing protocols and equipment are being used in the same sites, while multiple and
shifting hot zones are complicating matters further.
• Another hotly debated issue is the access to Prussian Blue for responders and their
families.
• Some members of the public feel that there is preferential treatment for these groups,
while those with access are raising questions about the side effects and long term
consequences.
• Occupational health and safety for traditional and non-traditional responders has
become a major issue in the context of the scenario, both in the short term and in the
longer term.
'5'&&66&5
105
• In order to address these concerns, please work both within your own pod and with the
other two pods for the following tasks outlined in the worksheet.
106
'5'&&66&5
1) Determine the two most significant responder health and safety issues that you feel are
most important for you to address. Record these issues in the form below. Then consult
with others in your pod and in other pods to understand their priority issues. Use the
form below to keep track of their issues.
YOUR PRIORITY HEALTH & SAFETY ISSUES:
HEALTH & SAFETY ISSUE #1:
HEALTH & SAFETY ISSUE #2:
'5'&&66&5
107
PRIORITY HEALTH & SAFETY ISSUES FROM OTHERS IN YOUR
POD
Organization:
Health & Safety Issue #1
Health & Safety Issue #2
Organization:
Health & Safety Issue #1
Health & Safety Issue #2
108
'5'&&66&5
PRIORITY HEALTH & SAFETY ISSUES FROM OTHERS IN
OTHER PODS
Organization:
Health & Safety Issue #1
Health & Safety Issue #2
Organization:
Health & Safety Issue #1
Health & Safety Issue #2
Organization:
Health & Safety Issue #1
Health & Safety Issue #2
Organization:
Health & Safety Issue #1
Health & Safety Issue #2
Organization:
Health & Safety Issue #1
Health & Safety Issue #2
Organization:
Health & Safety Issue #1
Health & Safety Issue #2
'5'&&66&5
109
2) In consultation with the other members in all the pods, determine where there are
overlaps, potential gaps or contradictions with respect to YOUR health and safety issues.
Record the overlaps and gaps in the form below.
YOUR HEALTH & SAFETY ISSUE #1:
Identified overlaps, gaps, and contradictions with other members` health and safety
issues:
YOUR HEALTH & SAFETY ISSUE #2:
Identified overlaps, gaps, and contradictions with other members` health and safety
issues:
110
'5'&&66&5
3) Work with the other members to decide on a plan for following up on these health and
safety concerns. Describe the actions that need to be taken for YOUR health and safety
issues to be addressed, taking into account options identified. Record the options and
decisions in the grid below.
YOUR HEALTH & SAFETY ISSUE #1:
Potential options for grouping or combining actions to be taken with other members:
Decision on grouping or not grouping actions with other members:
Rationale for the decision on grouping/not grouping and any changes to actions
required if grouping:
Decision on which actions will be implemented: (who, when, what methods)
YOUR HEALTH & SAFETY ISSUE #2:
Potential options for grouping or combining actions to be taken with other members:
Decision on grouping or not grouping actions with other members:
Rationale for the decision on grouping/not grouping and any changes to actions
required if grouping:
Decision on which actions will be implemented: (who, when, what methods)
'5'&&66&5
111
K.5
Closed Pod Collaboration Task Worksheet #1
INSTRUCTIONS
• First 55 minutes:
o Work with the members of your pod on the tasks outlined in this worksheet.
o Please record your responses directly on the worksheet – these will be collected
at the end of the session.
• Final 15 minutes:
o When instructed, complete the questionnaire that was included in the
envelope.
o Your completed questionnaire will be collected at the end of the session.
BACKGROUND INFORMATION FOR TASK
• Over the past 24 hours, there have been media reports that martial law and curfews are
going to be applied. Rumours are circulating that special legislation or the Emergencies
Act will be declared, giving military greater control of the city and any movement in or
out of it.
• While there is no truth to any of these rumours, there are armed and protected soldiers
who are guarding various sites and moving throughout the city to detect additional
sources of contamination; sightings of these soldiers are adding fuel to rumours.
• As a result, nervous citizens are evacuating prematurely, ignoring instructions to shelter
in place.
• There are increasing levels of criticism and concern from both citizens and local political
representatives that these communication issues have not been adequately addressed.
• Criticisms include conflicting instructions to the public, messages that are vague and
unclear, and major gaps in information.
• In order to address these criticisms, please work within your pod for the following tasks
outlined in the worksheet.
112
'5'&&66&5
1) Come to a consensus across all members within the pod on the two priority messages or
pieces of information that the group feels it is most important to relay to the general
public at this point. Record these messages in the form below.
MESSAGE #1:
MESSAGE #2:
'5'&&66&5
113
2) In consultation with the other members in the pod, identify two potential options for
approaches that could be used by the group to effectively deliver each of the two priority
messages to the public. In developing approaches, consider who will deliver the message,
when it can be delivered, what venue or method would be effective, and considerations to
take in account if the option was to be selected.
MESSAGE #1 – OPTION #1
Main messenger:
Timing of message:
Venue/method:
Considerations:
MESSAGE #1 – OPTION #2
Main messenger:
Timing of message:
Venue/method:
Considerations:
114
'5'&&66&5
MESSAGE #2 – OPTION #1
Main messenger:
Timing of message:
Venue/method:
Considerations:
MESSAGE #2 – OPTION #2
Main messenger:
Timing of message:
Venue/method:
Considerations:
'5'&&66&5
115
3) As a group, make decisions on which option for each message is most appropriate.
MESSAGE #1 DECISION
Decision:
Rationale for decision:
MESSAGE #2 DECISION
Decision:
Rationale for decision:
116
'5'&&66&5
K.6
Closed Pod Collaboration Task Worksheet #2
INSTRUCTIONS
• First 55 minutes:
o Work with the members of your pod on the tasks outlined in this worksheet.
o Please record your responses directly on the worksheet – these will be collected
at the end of the session.
• Final 15 minutes:
o When instructed, complete the questionnaire that was included in the
envelope.
o Your completed questionnaire will be collected at the end of the session.
BACKGROUND INFORMATION FOR TASK
• Responders travelling from other jurisdictions, contractors, volunteers as well as the
various unions for the professional responders are all questioning whether or not they
are being adequately protected and what standards for personal protective equipment
should be imposed.
• Differing protocols and equipment are being used in the same sites, while multiple and
shifting hot zones are complicating matters further.
• Another hotly debated issue is the access to Prussian Blue for responders and their
families.
• Some members of the public feel that there is preferential treatment for these groups,
while those with access are raising questions about the side effects and long term
consequences.
• Occupational health and safety for traditional and non-traditional responders has
become a major issue in the context of the scenario, both in the short term and in the
longer term.
• In order to address these concerns, please work within your pod for the following tasks
outlined in the worksheet.
'5'&&66&5
117
4) Come to a consensus across all members within the pod on the two most significant
responder health and safety issues that you feel are most important to address. Record
these issues in the form below.
HEALTH & SAFETY ISSUE #1:
HEALTH & SAFETY ISSUE #2:
118
'5'&&66&5
5) In consultation with the other members in the pod, identify two potential options for
approaches that could be used by the group to effectively address the two health & safety
issues. In developing approaches, consider who will deliver the interventions, when it can
be delivered, what method would be effective, and considerations to take in account if
the option was to be selected. Describe these approaches in the forms below.
HEALTH & SAFETY ISSUE #1 – OPTION #1
Who will deliver the interventions:
Timing:
Method:
Considerations:
HEALTH & SAFETY ISSUE #1 – OPTION #2
Who will deliver the interventions:
Timing:
Method:
Considerations:
'5'&&66&5
119
HEALTH & SAFETY ISSUE #2 – OPTION #1
Who will deliver the interventions:
Timing:
Method:
Considerations:
HEALTH & SAFTEY ISSUE #2 – OPTION #2
Who will deliver the interventions:
Timing:
Method:
Considerations:
120
'5'&&66&5
6) As a group, make decisions on which option for each health and safety issue is most
appropriate. Record the decisions and rationale in the grid below.
HEALTH & SAFETY ISSUE #1:
Decision on which option to act on:
Rationale for the decision:
HEALTH & SAFETY ISSUE #2:
Decision on which option to act on:
Rationale for the decision:
'5'&&66&5
121
K.7
Closed Pod Coordination Task Worksheet #1
INSTRUCTIONS
• First 55 minutes:
o Work with the members of your pod on the tasks outlined in this worksheet.
o Please record your responses directly on the worksheet – these will be collected
at the end of the session.
• Final 15 minutes:
o When instructed, complete the questionnaire that was included in the
envelope.
o Your completed questionnaire will be collected at the end of the session.
BACKGROUND INFORMATION FOR TASK
• Over the past 24 hours, there have been media reports that martial law and curfews are
going to be applied. Rumours are circulating that special legislation or the Emergencies
Act will be declared, giving military greater control of the city and any movement in or
out of it.
• While there is no truth to any of these rumours, there are armed and protected soldiers
who are guarding various sites and moving throughout the city to detect additional
sources of contamination; sightings of these soldiers are adding fuel to rumours.
• As a result, nervous citizens are evacuating prematurely, ignoring instructions to shelter
in place.
• There are increasing levels of criticism and concern from both citizens and local political
representatives that these communication issues have not been adequately addressed.
• Criticisms include conflicting instructions to the public, messages that are vague and
unclear, and major gaps in information.
• In order to address these criticisms, please work within your pod for the following tasks
outlined in the worksheet.
122
'5'&&66&5
4) Determine the two main messages or pieces of information that you feel that it is most
important for you to relay to the general public at this point. Record these messages in
the form below. Then consult with others in your pod to understand their priority
messages. Use the form below to keep track of their messages.
YOUR PRIORITY MESSAGES:
MESSAGE #1:
MESSAGE #2:
PRIORITY MESSAGES FROM OTHERS IN YOUR POD
Organization:
Message #1
Message #2
Organization:
Message #1
Message #2
'5'&&66&5
123
5) In consultation with the other members in your pod, determine where there are overlaps
in messages, potential gaps or contradictions with respect to YOUR messages. Record the
overlaps and gaps in the form below.
YOUR MESSAGE #1:
Identified overlaps, gaps, and contradictions with other members` messages:
YOUR MESSAGE #2:
Identified overlaps, gaps, and contradictions with other members` messages:
124
'5'&&66&5
6) Work with the other members in the pod to determine options for combining or grouping
the messages from the other pod members, and then decide on the most appropriate
grouping and methods of delivery for YOUR messages that need to be delivered to the
public taking into account options identified. Record the options and decisions in the grid
below.
YOUR MESSAGE #1:
Potential options for grouping/combining with other messages:
Decision on grouping or not grouping with other messages
Rationale for decision on grouping or not grouping and any changes required to message if
grouping:
Decision on how message will be delivered (who, when, venue/method)
YOUR MESSAGE #2:
Potential options for grouping/combining with other messages:
Decision on grouping or not grouping with other messages
Rationale for decision on grouping or not grouping and any changes required to message
if grouping:
Decision on how message will be delivered (who, when, venue/method)
'5'&&66&5
125
K.8
Closed Pod Coordination Task Worksheet #2
INSTRUCTIONS
• First 55 minutes:
o Work with the members of your pod on the tasks outlined in this worksheet.
o Please record your responses directly on the worksheet – these will be collected
at the end of the session.
• Final 15 minutes:
o When instructed, complete the questionnaire that was included in the
envelope.
o Your completed questionnaire will be collected at the end of the session.
BACKGROUND INFORMATION FOR TASK
• Responders travelling from other jurisdictions, contractors, volunteers as well as the
various unions for the professional responders are all questioning whether or not they
are being adequately protected and what standards for personal protective equipment
should be imposed.
• Differing protocols and equipment are being used in the same sites, while multiple and
shifting hot zones are complicating matters further.
• Another hotly debated issue is the access to Prussian Blue for responders and their
families.
• Some members of the public feel that there is preferential treatment for these groups,
while those with access are raising questions about the side effects and long term
consequences.
• Occupational health and safety for traditional and non-traditional responders has
become a major issue in the context of the scenario, both in the short term and in the
longer term.
• In order to address these concerns, please work within your pod for the following tasks
outlined in the worksheet.
126
'5'&&66&5
4) Determine the two most significant responder health and safety issues that you feel are
most important for you to address. Record these issues in the form below. Then consult
with others in your pod to understand their priority issues. Use the form below to keep
track of their issues.
YOUR PRIORITY HEALTH & SAFETY ISSUES:
HEALTH & SAFETY ISSUE #1:
HEALTH & SAFETY ISSUE #2:
PRIORITY HEALTH & SAFETY ISSUES FROM OTHERS IN YOUR
POD
Organization:
Health & Safety Issue #1
Health & Safety Issue #2
Organization:
Health & Safety Issue #1
Health & Safety Issue #2
'5'&&66&5
127
5) In consultation with the other members in your pod, determine where there are overlaps,
potential gaps or contradictions with respect to YOUR health and safety issues. Record
the overlaps and gaps in the form below.
YOUR HEALTH & SAFETY ISSUE #1:
Identified overlaps, gaps, and contradictions with other members` health and safety issues:
YOUR HEALTH & SAFETY ISSUE #2:
Identified overlaps, gaps, and contradictions with other members` health and safety issues:
128
'5'&&66&5
6) Work with the other members to decide on a plan for following up on these health and
safety concerns. Describe the actions that need to be taken for YOUR health and safety
issues to be addressed, taking into account options identified. Record the options and
decisions in the grid below.
YOUR HEALTH & SAFETY ISSUE #1:
Potential options for grouping or combining actions to be taken with other members:
Decision on grouping or not grouping actions with other members:
Rationale for the decision on grouping/not grouping and any changes to actions required if
grouping:
Decision on which actions will be implemented: (who, when, what methods)
YOUR HEALTH & SAFETY ISSUE #2:
Potential options for grouping or combining actions to be taken with other members:
Decision on grouping or not grouping actions with other members:
Rationale for the decision on grouping/not grouping and any changes to actions required if
grouping:
Decision on which actions will be implemented: (who, when, what methods)
'5'&&66&5
129
Annex L
Consistency of Interpretation Instrument
This questionnaire is designed to assess the scenarios with respect to which facts can be recalled.
Please select the best answer for each question. Circle one response for each question.
1) Where did the initial package explode?
a. Hospital loading dock
b. International Borders and Customs warehouse
c. Sort and Send Mail Delivery warehouse
d. GAP TV Station
2) Approximately how many people arrived at the examination centers set up in the
hospital parking lot within the first 24 hours?
a. 10,000
b. 5,000
c. 1,000
d. 200
3) Within 24 hours, the radius evacuated around the explosion was:
a. 500m
b. 2km
c. 10km
d. 20km
4) The bomb was assembled:
a. In a suburban house
b. In a small town outside of GAPville
c. In the parking lot by the hospital
d. In a van outside the warehouse
130
'5'&&66&5
5) The number of houses that need to be decontaminated or destroyed:
a. 20
b. 100
c. 300
d. 500
6) The spokesperson for the situation report, Mr. Robert Brights, has the title:
a. National Emergency Manager, CBRN Team
b. Chief of Police, Gapville
c. Chief of Liaison, City of Gapville
d. Director of Emergency Management, Unified Coordinating Command
7) Additional chemical protective gear for responders and volunteers was obtained from:
a. Red Cross
b. Pulp and Paper Mill
c. National CBRN Team
d. National Firefighters Association
8) What was the intended target for the bomb?
a. Hospital
b. Entertainment district
c. Airport
d. GAP TV Station
9) At the 48-hour mark, which of the following modes of transport are still partially
operational?
a. Light rail
b. Bus lines
c. Airport
10)Which of the following sites is not contaminated:
a. South End LRT Station
b. Airport
c. Mosque
d. Johnson Estates
'5'&&66&5
131
Annex M Scenario Assessment Guide Contents
M.1
Scenario Assessment Guide: Scenario Complexity Rating
Instructions
Estimated Time to Complete: 2 hours (40 minutes per scenario)
Steps:
1. Please review the Information and Consent Form located on the following page. If you
agree to its contents, please sign the form.
2. Fill out the Background Information Questionnaire.
3. Read the definitions given for simple, complicated and complex.
4. Read the first scenario provided in this guide.
5. Considering the scenario, fill out the Scenario Assessment Questionnaire for the first
scenario. Note: for ease of reference, you may want to remove the questionnaire from the
package and place beside the scenario.
6. Read the second scenario provided, and complete the questionnaire.
7. Repeat with the third and final scenario.
8. Return all scenarios and completed questionnaires to the facilitator.
Thank you for your assistance.
132
'5'&&66&5
M.2
Scenario Assessment Guide: Confidentiality / Intellectual
Property Agreement
Subject:
Agreement for the University of Ottawa project entitled “In Vivo Simulation
of Meta-Organizational Shared Decision Making” (the “Project”)
In order to protect confidentiality of materials created and data collected over the course of the
“In Vivo Simulation of Meta-Organizational Shared Decision Making” project, we are kindly
asking reviewers of materials to agree to the confidentiality and intellectual property clauses
described below.
Confidentiality:
1.1
“Confidential Information” means all data and information relating to the research
project.
1.2
I understand and agree that, as part of my participation, I may have access to
Confidential Information belonging to the University of Ottawa or other participants
including, without limitation, the results of this project. I will regard such Confidential
Information as strictly confidential and agree not to reveal any such Confidential
Information to any other person, firm, corporation, company or entity unless specifically
authorized to do so in writing by Louise Lemyre, the Principal Investigator of the
Project. This obligation of secrecy will continue after my participation in the Project has
ended.
Intellectual Property:
2.1
“Intellectual Property” includes, but is not limited to: technical information, know-how,
copyrights, patents, models, patterns, drawings, specification, prototypes, and inventions.
2.2
I agree to assign ownership of all Intellectual Property I contribute to create, develop or
produce as part of my participation in the Project to the University of Ottawa. I agree to
execute such additional conveyances and other documents conveying title or copyright to
such Intellectual Property to the University of Ottawa as the University of Ottawa may
require. Should any Intellectual Property assigned to the University of Ottawa be in the
form of copyrights, as an author of copyrighted Intellectual Property, I hereby
permanently waive my moral rights, as defined in the Copyright Act, in respect such
Intellectual Property.
Signed:
______________________________
Name:_________________________
'5'&&66&5
____________________
(Witness)
Date:____________________
133
M.3
Scenario Assessment Guide: Background Information
Questionnaire
10. :KDWLV\RXUFXUUHQWMREWLWOH"BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
11. +RZORQJKDYH\RXUZRUNHGLQWKLVSRVLWLRQ"BBBBBBPRQWKVBBBBBBB\HDUV
12. +RZORQJKDYH\RXZRUNHGIRUWKLVRUJDQL]DWLRQ"BBBBBBBPRQWKVBBBBBBB\HDUV
13. +RZORQJKDYH\RXZRUNHGLQHPHUJHQF\PDQDJHPHQW"BBBBBBPRQWKVBBBBBB
\HDUV
14. :KDWDUH\RXUPDLQUROHVDQGUHVSRQVLELOLWLHVZLWKUHVSHFWWR
a. 3ODQQLQJIRUHPHUJHQFLHV
b. 5HVSRQGLQJWRHPHUJHQFLHV
15. :KDWLV\RXUDFDGHPLFEDFNJURXQG"
a. &ROOHJHGLSORPDV
b. 8QLYHUVLW\GHJUHHV
16. :KDWLV\RXUFXUUHQWDJH"BBBBBBB\HDUV
17. *HQGHU
18. :KLFKODQJXDJHVDUH\RXIOXHQWHQRXJKLQWRZRUNLQ"&KHFNDOOWKDWDSSO\
† French
† English
† Other
If Other, please list:
134
'5'&&66&5
M.4
Scenario Assessment Guide: Definitions of Simple,
Complicated and Complex
Simple, complicated and complex can be conceptualized as a continuum ranging from simple to
complex, with complicated located at the midpoint of the continuum.
Simple Æ Complicated Æ Complex
Simple situations can be defined as those that are well understood, and involve a minimal number
of organizations to resolve satisfactorily. A simple situation is well understood and thus each
organization follows standard operating procedures. In simple situations, solutions are known and
the scope of the situation is within an organization’s capability.
Complicated situations are characterized as generally well understood, but they involve multiple
interrelated organizations. One main difference between simple and complicated situations is the
number of organizations required to resolve the situation, and the dynamic nature of the situation.
The main similarity is that the situation is relatively well understood, including the cause and
effect relationships between actions and outcomes. The situation, including the effect of
interventions, is relatively well understood. In complicated situations, the solutions are known
but the scope is beyond one organization’s capability.
Complex situations involve considerably fewer “knowns” than either simple or complicated
situations. Due to the high level of uncertainty, the cause and effect relationship between actions
and outcomes becomes unclear. Also characteristic of complex situations is that standard actions
or procedures that might be commonly used in less complex situations can cause large ripple
effects that then further contribute to the complexity of the situation. In complex situations, the
solution is partly unknown and the scope is well beyond one organization’s capability.
'5'&&66&5
135
M.5
Scenario Assessment Guide: Blackout Scenario
Wednesday 6:00 am
Wednesday morning, the province of Anticipate is covered in ice after one night of
unexpected heavy frozen rain. The hardest hit city of Gapville wakes up to over 43 mm
of ice build-up, strong winds, plummeting temperatures of minus 20 degree Celsius
weather and a power outage across the entire city. Traffic lights aren't working, bank
machines, point-of-sale purchases and gas pumps are inaccessible. Public transit is
paralyzed with the weather conditions and the blackout.
The municipal government orders all snow removal teams onto the roads, but they are
overwhelmed, and need support. Gapville Hydro workers are trying to find the source of
the city-wide power outage, and are repairing the ice damaged lines as fast as possible.
Wednesday 12:00 pm
The McLaughlin Memorial Hospital is facing serious challenges. The back-ups power
systems have failed for unknown reasons. Without power, or working back-up systems,
critical patients are in dire need. The emergency room is already becoming busy with
increased accidents due to the icy conditions, and vulnerable patients with exposure to the
cold from a morning with no heat. The head of the hospital contacts the municipal
government, and asks for help transferring and sheltering patients.
Wednesday 2:00 pm
The Gapville municipal government asks the local Fire and Police departments to call in
all available personnel to deal with the slippery streets, and the overflow in the crippled
hospital. Mobilizing the municipal forces is difficult due to the chaotic state on the roads.
It takes several hours for help to arrive on the streets and in the hospital.
Wednesday 6:00 pm
Within twelve hours, Gapville Hydro workers have managed to restore the power to a
limited section of the city. However, hydro workers are beginning to find suspicious
activity not indicative of a power failure. Notably, backup systems for the McLaughlin
Memorial Hospital and the Gapville Transit system are still not operational. These
backup systems run on generators, an independent power supply, and should technically
not be affected by the power outage. Gapville Hydro arrives with additional generators at
the Hospital, and attempt to hook-up the network. Again, efforts fail, power cannot be
restored, and patients must be evacuated.
Facing the first night with no heat or light, many of Gapville’s local community networks
begin to mobilize their own resources, walking to neighbours who may be vulnerable,
and opening the community centers and churches, offering shelter for those who need to
136
'5'&&66&5
get warm. Fuel is an issue, for building generators, and the portable generators being
supplied by local citizens.
Municipal road crews are still hard at work, as is Gapville Hydro, clearing roads and
repairing damaged power lines. EMS is now also working with health care workers to
move patients from the Hospital to the local community center, which has been outfitted
with an industrial sized generator donated from the local lumber yard. There are
concerns that fuel for the generator will only last one night.
Thursday 7:00 am
The municipal government contacts the Province, asking them to request the assistance of
the Armed Forces to provide specialized equipment to help with the road clearing and
continuation of vital services. The mayor also asks for additional fuel and generators to
be brought in. The military agree to allocate some of their resources to Gapville, and
announce they will be arriving in two days. As the ice-storm affected most of the
province of Anticipate, the military has several sites they need to attend to, and interprovincial travel is still slow.
Thursday 1:00 pm
After working around the clock for over 31 hours, Gapville Hydro workers discover the
back-up system failures at the Hospital and the Transit System are due to computer
issues, and not the downed power lines. Hydro sends their computer analysts to the
McLaughlin Memorial Hospital and the GapTransit main street terminal, to try to reboot
the back-up emergency systems.
Thursday 5:00 pm
The community center and local churches are becoming crowded with people seeking
heat and light, for the second night of the power outage. With the blackout there is
limited access to television or internet for information. Battery operated radios are used to
collect any information available. Due to the blackout, the local radio station is still
unable to broadcast using the GapTV tower, so the only news available is from the
National Radio Service. The information is broad, and community specific information is
hard to come by. The only specific information for Gapville is that Hydro Company is
still unable to provide a time-frame for restoration of power.
Thursday 10:00 pm
Volunteers and Emergency Response agencies are doing what they can to identify
vulnerable groups, and to get to those most in need of help. The senior’s home is in the
blackout area, and exposure to cold is becoming an increasing concern. Many private
groups have banded together to help, providing fuel, generators, hand crank cell phone
chargers, blankets and candles. Several people have been found using incredibly
dangerous practices for heat, for example, using gas powered generators and BBQ’s
indoors.
'5'&&66&5
137
Friday 9:30 am
Hydro authorities contact Communications Co., the software and hardware suppliers for
the systems at the hospital and the transit system. Gapville Hydro explains that they have
been unable to reboot the back-up systems, and there is still no power supply at the
Hospital or at the Transit System. The Communications Company sends senior analysts
to the power plant.
Friday 12:00 pm
Power has been restored at GapTV, where the city’s communication tower resides.
Broadcast resumes in a limited for local television and radio. Gapville Hydro expects it
will take another two days for power to fully return to the city. The lack of information is
beginning to cause the formation of rumours. With two nights of no heat or light, citizens
are getting frustrated, and are wondering when the power will be back online. There have
been rumours of a fire at the Hydro Station that will make power restoration incredibly
difficult, and could take up to a month to fix. Many people are planning larger scale
evacuations due to this misinformation, some are planning on staying with family in other
cities and many are hoping that commuter trains, or flights will soon be running. The
train station and airport are beginning to see a lot of people arriving frustrated with plans
of evacuation and no way to carry them out.
INJECT – Task #1
Saturday 8:00 am
The Armed Forces arrive in Gapville with road clearing materials, supplies for shelters,
generators and additional fuel supplies. The arrival of the military provides some relief to
the distressed citizens. Gapvillians feel that their Mayor has taken some steps to ensure
their safety, but are still frustrated with the lack of communication.
Saturday 2:30 pm
The Communications Company discovers the source of the computer failure was due to
compromised hardware. The Hospital’s computers and backup safety systems were built
using counterfeit microchips which allow backdoor access to hackers. According to the
serial numbers, the compromised hardware arrived in a large shipment last year. This
revelation also takes the power failure from an accident to an attack. When this
revelation is made, the Reframe Federal Government is alerted, and investigators from
the federal cyber task force are sent to Gapville. They should arrive by Monday morning.
Sunday 12:00 pm
The combined efforts of fire police and emergency services, working alongside the
military, volunteer services, municipal crews, and Gapville Hydro has made serious
headway into road clearing and power restoration. Over a quarter of the city has clear
roadways and power. Several churches and public buildings can now be safely used for
shelter, with heat and light.
138
'5'&&66&5
Monday 11:00 am
Private analysts track down all the infected computer systems in the city. In total, three
infected systems are identified. The first identified is at the McLaughlin Memorial
Hospital. The newly upgraded computer patient tracking systems are compromised.
These systems run on a national E-Health network, and there is concern that confidential
medical information all across the country of Reframe has been hacked. The local
GapTransit system has compromised hardware in its recently installed navigation
systems. And finally, the National Bank switched to new teller and ATM software 6
months ago, for all of their branches across the province.
Monday 2:00 pm
The potential size and scope of the computer recall, and the damage expected to the city's
infrastructure during recalls and retrofitting is overwhelming. The obviously malicious
and well-coordinated computer infiltration has federal investigators concerned it is a
terrorist attack.
The freezing rain and ice pellets end and the skies clear. It is still very cold, at -22
degrees Celsius. The change in weather allows municipal road workers and the military
to get ahead on the road clearing. Hydro workers are also able to get almost half of the
city's power up and running.
Monday 4:30 pm
Communications Co. send specialists to the McLaughlin Memorial Hospital, to replace
their faulty systems. The hospital is still without main power, but once the backup
computer systems are restored, the generators are functioning, the ICU and Emergency
Room have limited power, and there are limited lights, but there is heat. It will still take
time, however, to get the patient tracking and E-Health networks secured.
Tuesday 9:00 am
The Federal Government, the Gapville Municipal government and P-RAM
communications hold a press conference at GapTV for those residents with power. The
Gapville Mayor explains that power is coming back on-line, but will not be fully restored
for two more days. The weather forecasts are looking clear for the next week, and should
allow the road crews and military to complete their work on clearing the city within the
next few days as well. The mayor provides a list of available shelters, and some
recommendations for health and safety during this difficult time. The federal cyber task
force investigators then explain that the power failures are not fully caused by the ice
storm. There were computer failures at the Hospital, the GapTransit System and the
National Bank, caused by malicious compromised hardware. The investigators explain
that embedded chips are built into the computers, and allow access to vital infrastructure
computer systems. Because of these hacked computer chips, back-up emergency systems
failed, and have created a devastating breach of privacy in health and financial records,
nationwide. The government asks people to remain calm, and not to attempt to withdraw
'5'&&66&5
139
their savings from the banks. Officials ask for patience, and trust that they will be able to
correct these vital systems soon.
Communications Co. explains they were victims of counterfeit hardware purchase. Due
to the recall of these systems, the banking system will be off-line until security can be
ensured, and the transit system will not be operational for at least a week. The hospital
has backup systems functioning, and full power will be restored soon, but all nonessential procedures will be suspended until the E-Health network can also be secured.
Tuesday 2:00 pm
Gapvillians are all reeling from the breadth and scope of the cyber attack on their nation.
The entire country feels the effect of the largest national bank going off-line, and the
province is still mostly covered in ice. The official press conference has left people
feeling vulnerable and scared. There is a run on the banks, with many people attempting
to withdraw their savings. But with systems down, the tellers are unable to complete any
transactions. As people become more frightened, security concerns arise, and police are
called away from their efforts in the hospitals and on the ice-clogged streets to guard the
banks, and keep some order.
Tuesday 6:00 pm
The city is preparing for another night with no heat in many areas. Roads are still blocked
in many smaller streets, but the main thoroughfares and the highway have been cleared.
While the municipal road crews finish the clearing the military turns to helping with
shelter for the over 100,000 people still without power. At the same time, the private
sector analysts and federal investigators are working hard on the first recalls, and tracking
down the source of the cyber-attack.
Task #2
140
'5'&&66&5
M.6
Scenario Assessment Guide: Scenario Assessment
Questionnaire – Blackout Scenario
This form is designed to capture a reader’s understanding of what contributes to the complexity of a given
scenario. Potential contributions to complexity for a scenario are described below in terms of impact,
uncertainty and vulnerability, with specific challenges or areas noted for each. After reading the scenario,
the reviewer is asked to rate the degree to which each challenge was evident in the scenario. The results
will be compared with data collected from other reviewers to select a suitably complex scenario for use in a
simulation exercise.
Q1 Overall, which of the following descriptors best describes the scenario?
† Simple
† Complicated
† Complex
Please rate the extent to which each of the following challenges is evident in the scenario. Use the scale
ranging from 0 (not at all evident) to 4 (strongly evident).
0
1
2
3
4
Impact
Q2
Q3
Q4
Q5
Q6
Impact severity
(i.e., number of fatalities /casualties,
amount of physical damage/ psychological
trauma, degree of social disruption/multiple
events)
Impact scope
(i.e., breadth of impact on people, services,
structures, societal functions )
Impact timing
(i.e., no threat/warning phase, sudden
onset, sustained duration, and/or recurring
event)
Media involvement
(i.e., extensive media coverage, lack of
access to official sources, inappropriate
use of social media tools)
Political processes
(i.e., multiple organizations involved,
multiple jurisdictions, and/or multiple levels
of government, public fear and/or mistrust
in officials)
Uncertainty
'5'&&66&5
141
0
Q7
Q8
Q9
Q10
Q11
Q12
1
2
3
Novelty of situation
(i.e., novel hazard, malicious cause, lack of
experience, lack of controllability, and/or
latent impacts)
Anticipation and planning
(i.e., lack of planning, rehearsal, and
sharing of plans, and/or lack of integration
of plans)
Data and information
(i.e., lack of access to information,
inaccurate information, lack of credible
information sources, excessive volume of
data, and/or fast speed of information flow)
New organizations and partners
(i.e., presence of non-traditional
organizations and partners, and/or
presence of emergent organizations or adhoc groups that self-organize to fill
perceived resource gaps)
Changing context
(i.e., big changes in situation context, fast
speed of change, actual/perceived
negative consequences of change, and/or
negative impacts of intervention (negative
secondary effects)
Flexibility of interpretive frameworks
(i.e., imposed/unilateral processes, lack of
common/shared processes, and/or lack of
creativity/improvisation)
Vulnerability
Q13
Q14
Q15
142
Economic development
(i.e., lack of resources, lack of economic
diversity, unequal distribution of wealth,
and/or economic instability)
Social capital
(i.e., large number of at-risk populations,
lack of social support, poor citizen
participation in volunteering and decision
making, poor sense of community, and/or
excessive/insufficient attachment to place)
Community competence
(i.e., political instability, poor leadership
skills, low literacy and education levels,
lack of experience with similar hazards,
and/or lack of community partnerships and
support)
'5'&&66&5
4
0
Q16
Q17
1
2
3
4
Information and communication
(i.e., unreliable or excessive amounts of
extraneous information, public opinion
ignored, lack of trusted media
spokesperson, ineffective risk messaging,
and/or poor communication infrastructure
capabilities)
Other infrastructure
(i.e., high degree of interdependence
among critical systems, lack of redundant
or backup systems, and/or weak or
blocked distribution networks)
Q18 In your estimation, which of the following factors contributed most to the level of complexity in the
scenario? Please check one box and explain your choice.
† Impact
† Uncertainty
† Vulnerability
† All of the above (factors influenced the assessment equally)
Please explain why:
Please rate the overall level of complexity of the scenario on a scale of 0 (not at all complex) to 4 (very
complex).
0
Q19
1
2
3
4
Overall, I would describe the level of
complexity for the scenario as:
Please rate your level of agreement with the following statements on a scale of 0 (strongly disagree) to 4
(strongly agree).
0
Q20
Q21
Q22
1
2
3
4
The first part of the scenario is realistic /
plausible.
The second part of the scenario is realistic
/ plausible.
The timeline for the scenario is realistic /
plausible.
'5'&&66&5
143
M.7
Scenario Assessment Guide: Mail Scenario
Monday 4:00 pm
There is a small explosion and fire at the Sort & Send Mail Delivery (SSMD) warehouse
in Gapville. The employees put out the fire with a couple of fire extinguishers while
calling 911. There are approximately 50 other employees in the building.
Three employees are injured from the event. The employee who was handling the
package at the time of explosion has more serious injuries. The two employees with
minor injuries are given first aid and taken by another employee to the hospital. They
don’t want to wait for the ambulance and don’t feel the need for emergency treatment.
They are seen for workers’ compensation reasons and released from the hospital. All of
the remaining employees in the facility stay at the warehouse and continue working.
The Gapville fire department arrives and gives initial medical care to the more seriously
injured employee. The paramedics then arrive to take over his care. They transport him to
the hospital trauma centre where he is seen in the emergency department and, given his
injuries, is admitted to the hospital for treatment of burns and a fractured arm. The
ambulance crew returns to service and responds to other calls throughout the city. The
fire department ensures that there is no further damage or fire and calls the Gapville
police department since it appears to be a suspicious fire. The fire crews return to their
stations. The police determine from SSMD records that the package had been dropped
off early in the day and was to be delivered to the GapTV television station the next day.
Monday 7:00 pm
All of the emergency personnel in Gapville - EMS, Fire, Police and Hospital staff
complete shift changes between 4:00 pm and 6:00 pm. Those responders who were on the
scene of the SSMD explosion now are off-duty, and travel to home, schools, restaurants
and other recreational activities.
Gapville Police Station #2 receives an anonymous call, announcing that a dirty bomb has
been sent to the GapTV television station. The police connect the call with the situation at
the SSMD warehouse and call the HazMat team, Federal Investigators, the Military, and
the Provincial Government. If this caller’s information is true, this revelation takes the
SSDM explosion from a potential accident to a radiological attack. The military announce
they will arrive within 72 hours.
Monday 10:00 pm
After a brief investigation, the HazMat team determines that the explosion spread
approximately one ounce of Cesium 137 in the building. Initial readings of the exposure
level at the site were 100 rems/hour; however, those who were one meter from the
144
'5'&&66&5
explosion received a dose of 1000 rems/hr and those 10 meters away, between 100 and
543 rems/hr. Those who handled the patient were exposed to slightly higher dosages
because the dust came off of the patient’s clothing and transferred to them. Other hospital
staff and patients in the treatment area of the emergency room received between 10-15
rems/hour.
The explosion was contained for the most part in the SSDM facility and all those who
had been there were exposed and contaminated to varying degrees. Employees had used
fire extinguishers to combat the flames, and the force of extinguishers spread the Cesium
137 around the ventilation ducts in the warehouse. All responders were also exposed and
some experience some degree of radiation sickness over the coming days depending on
how close they were to the source and the patient.
In addition, for those in the facility, the long term risks of cancer increased significantly
as a result of exposure to inhalation of contaminated dust particles. Employees, police,
EMS and fire fighters received a dose of radiation that increased their risk of cancer to 1
in 5. Those in the hospital emergency room increased their cancer risk to 1 in 50. An
additional risk was created by the dust particles on the clothing and equipment that was in
taken home, or into the public, by individuals. This increased the cancer risk in areas of
the public to about 1 in 100.
At about the same time that the HazMat team confirmed the presence of Cesium 137, the
SSMD employee who was hospitalized earlier begins to show initial symptoms of
radiation exposure and his condition becomes critical. When notified by the police, the
hospital calls a “code brown”, for hazardous materials, and closes the emergency
department to all incoming cases. After three hours, all three injured SSMD employees
begin to show signs of radiation sickness. The most seriously injured patient is now
critically ill while the others, who are at home, are less ill but are showing signs of nausea
and red skin.
Tuesday 9:00 am
Tracking is underway for all those who were in contact with the SSMD patients or who
were at the scene of the event. In addition, Public Health authorities are tracking any
people who were at the ER when the SSMD patients came in. Working in conjunction
with GapTV, the authorities hold a press conference. They ask that anyone who was in
the exposed buildings, or near an affected victim to go to the hospital to get checked out.
The municipal authorities have set up temporary exam centers in the parking lot of the
McLaughlin Memorial Hospital. All who were directly exposed will become ill within 72
hours and will be showing different signs of radiation exposure over the coming weeks.
Tuesday 7:30 pm
Hospital staff has worked overtime, dealing with the influx of patients, coupled with the
loss of employees who have been exposed to radiation.
'5'&&66&5
145
Hospital authorities request help from the neighbouring city’s Hospital. There is
confirmation that the other SSMD employees, the paramedics and the initial ER staff who
treated the SSMD employees were exposed to radiation and with treatment will survive,
but with unknown impacts. All of the fire fighters, EMS and police at the scene were
also exposed to varying degrees of radiation. At the moment, 22 people are hospitalized
for treatment of sub-lethal but serious effects of radiation exposure; another 65 are
following out-patient treatments. Thousands have arrived at the examination centers set
up in the hospital parking lot. No one can estimate however, the degree of exposure and
the risk for each, either in the short or long term. Family members for most of the
responders have been found to have varying levels of exposure as a result of the
contaminated clothing brought home. Potentially exposed children are now the focus of
intensive surveillance and monitoring. It is estimated that the risk for cancer may be as
high as 1 in 50 for some of them.
Wednesday 9:00am
GapTV broadcasts another press conference. Authorities announce there have been no
further bombs or threats. Sadly, there is one confirmed death, the critically injured SSMD
employee, as a result of acute radiation exposure. Due to the amount of contamination,
the employee was buried in a lead coffin sealed in concrete. It is likely that the two
SSMD employees who initially exhibited minor physical injuries will also pass away.
Federal investigators have arrived and announce they have yet to track down the source
or motives for this attack. The public is scared and uncertain as to the source of the
radiological attack, and are very concerned about their exposure and their children’s
exposure to radiation. Equally there is a loss of public trust. How could authorities have
allowed contaminated responders to go out into the public after their shift? Why was this
not caught in time?
Officials are unable to give full exposure estimates but have taken a variety of actions
intended to restore public confidence. The fire stations that went to the fire have been
closed and their equipment and vehicles are being decontaminated. The ambulance, the
hospital emergency room (ER) and the rooms where the patients were handled are also
being decontaminated. The hospital itself has completely shut down and moved patients
to other facilities.
Wednesday 5:30pm
The police investigation identifies those responsible and where they assembled the bomb.
They have so far eluded the police, and it is thought they may have fled the country.
There is concern that they possibly contaminated themselves during the bomb assembly,
and that they may be spreading the contamination as they travel. The authorities have not
been able to determine how they left or where they went.
The perpetrators used a suburban house, which is now contaminated. The area for several
blocks around it is evacuated and sealed off while authorities start to conduct additional
tests. As they conduct these tests, they discover a natural source of radon which creates
146
'5'&&66&5
an additional concern for the people living in that area. There is a lot of confusion about
different sources and the public is not distinguishing between types of radiation but are
insisting that they be protected from any type.
Thursday 8:00am
There have been no further bombs or threats; however, there has been additional loss of
life. Over 15 people have died. Local funeral homes have stated that they do not want to
handle the bodies for fear of contamination. The families are upset, believing they are not
receiving adequate information or support. There is inconsistent media coverage with
different “experts” recommending a wide variety of methods for handling the sick and
deceased, including burial in lead coffins.
Thursday 3:30pm
The military arrive, and help with many different needs, moving critical patients from the
shopping mall to more appropriate treatment centers, such as the walk-in clinics and the
assisted living homes. They also set up additional evacuation centers, for those displaced
from the affected area, and set up decontamination sites at many different locations
across the city.
Thursday 5:00pm
Family members for all of the responders are found to have varying levels of exposure as
a result of the contaminated clothing being brought home. Children are the focus of
intensive surveillance and monitoring. Media reports have suggested that they will have a
long-term increased risk of developing cancer, which creates some public anxiety and
demand for more testing and information. There is significant internet chatter regarding
the long-term outcomes of this event.
Task #1
Friday 10:15 am
Each day, people turn up at the hospital asking to be screened and treated, insisting that
they have been exposed or manifesting symptoms of radiation exposure. Additionally,
people are demanding to know how long-term monitoring will be carried out, and in what
way will they be treated and/or compensated if they develop cancer.
There is considerable scepticism that the estimate of those exposed is accurate; many
suspect the number is being downplayed by authorities. Thus, there is a growing demand
that Prussian Blue, the only known treatment, to be widely distributed to the population,
even to anyone who might have been remotely exposed to the contamination. Prussian
Blue is not licensed for use in this country and requires a special license from Public
Health for this purpose. Officials are reluctant to provide anything other than limited
access. There is not enough Prussian Blue available to provide widespread treatment, so
only those first responders with confirmed exposure have been given the treatment. The
first responders demand that their family members be provided access also. The SSMD
'5'&&66&5
147
employees, who suffered just as much radiation exposure as the first responders, demand
treatment both for themselves and for their families.
Saturday, 12:00pm
Monitoring of people living in the neighbourhood where the bomb assembly took place
has not found any people with exposure to radiation. Officials, however, are not able to
reassure people that there will be no long-term negative effects. Those who have been
evacuated are demanding assistance in covering their costs, as well as compensation for
their homes which they say will not sell any time in the foreseeable future.
The SSMD site has been sealed for both decontamination and the police investigation.
Other businesses in the area have shut down, and tests are being conducted to determine
if there has been any leakage from the SSMD facility. Traffic in the area remains closed
which creates a number of problems for businesses in the vicinity. Again, military
support is needed to handle the flow of traffic, enforce the evacuation zone, and provide
shelter and support for those displaced persons. The military work with local authorities,
and volunteer agencies to try to mitigate some of the more unpleasant effects of
evacuation and decontamination.
Monday 9:00am
It is announced that the SSMD facility will have to be demolished along with all of the
material that was in it. All of the packages and documents in the facility have to be
destroyed. This creates problems for some of the companies affected. Insurance claims
pour in, and SSMD is struggling to maintain its customer base across the country. Courier
services in general have seen a significant drop in business.
Announced plans to re-open the fire station and the hospital are met with strong public
opposition. The public demands assurance that the decontamination efforts will be 100%
effective and that there is no further risk of exposure.
Thursday 9:00am
There are a number of lawsuits being filed against the emergency services for not
protecting the public and further endangering people in the vicinity of the station and
along the routes that they traveled on. The hospital has a series of legal actions claiming
that people died as a result of the decision to close and transfer them.
There are numerous issues related to the removal of contaminated material – i.e., where it
will go and how it will be disposed of. This has caught the media’s attention as well as
that of numerous groups who do not want it transported along city streets to wherever it
will finally end up. It has been estimated that the main site clean-up will take 6 to 8
months. There are conflicting reports about the risks posed to the community during this
activity. There is also a debate about how much of an area should be cleaned up – 2
blocks or 10 blocks around each of the different sites. The amount of waste from the
148
'5'&&66&5
SSMD warehouse is estimated to fill a football field waist deep. There is a suggestion
that the entire area be abandoned and closed off for 20 to 30 years.
Clean-up costs are estimated to be in the billions of dollars. There is an ongoing issue
about which level of government will be responsible for what costs. Insurance companies
are claiming that since it was an act of terrorism, any policies are null and void.
Task #2
'5'&&66&5
149
M.8
Scenario Assessment Guide: Scenario Assessment
Questionnaire – Mail Scenario
This form is designed to capture a reader’s understanding of what contributes to the complexity of a given
scenario. Potential contributions to complexity for a scenario are described below in terms of impact,
uncertainty and vulnerability, with specific challenges or areas noted for each. After reading the scenario,
the reviewer is asked to rate the degree to which each challenge was evident in the scenario. The results
will be compared with data collected from other reviewers to select a suitably complex scenario for use in a
simulation exercise.
Q1 Overall, which of the following descriptors best describes the scenario?
† Simple
† Complicated
† Complex
Please rate the extent to which each of the following challenges is evident in the scenario. Use the scale
ranging from 0 (not at all evident) to 4 (strongly evident).
0
1
2
3
Impact
Q2
Q3
Q4
Q5
Q6
Impact severity
(i.e., number of fatalities /casualties,
amount of physical damage/ psychological
trauma, degree of social disruption/multiple
events)
Impact scope
(i.e., breadth of impact on people, services,
structures, societal functions )
Impact timing
(i.e., no threat/warning phase, sudden
onset, sustained duration, and/or recurring
event)
Media involvement
(i.e., extensive media coverage, lack of
access to official sources, inappropriate
use of social media tools)
Political processes
(i.e., multiple organizations involved,
multiple jurisdictions, and/or multiple levels
of government, public fear and/or mistrust
in officials)
Uncertainty
150
'5'&&66&5
4
0
Q7
Q8
Q9
Q10
Q11
Q12
1
2
3
4
Novelty of situation
(i.e., novel hazard, malicious cause, lack of
experience, lack of controllability, and/or
latent impacts)
Anticipation and planning
(i.e., lack of planning, rehearsal, and
sharing of plans, and/or lack of integration
of plans)
Data and information
(i.e., lack of access to information,
inaccurate information, lack of credible
information sources, excessive volume of
data, and/or fast speed of information flow)
New organizations and partners
(i.e., presence of non-traditional
organizations and partners, and/or
presence of emergent organizations or adhoc groups that self-organize to fill
perceived resource gaps)
Changing context
(i.e., big changes in situation context, fast
speed of change, actual/perceived
negative consequences of change, and/or
negative impacts of intervention (negative
secondary effects)
Flexibility of interpretive frameworks
(i.e., imposed/unilateral processes, lack of
common/shared processes, and/or lack of
creativity/improvisation)
Vulnerability
Q13
Q14
Q15
Economic development
(i.e., lack of resources, lack of economic
diversity, unequal distribution of wealth,
and/or economic instability)
Social capital
(i.e., large number of at-risk populations,
lack of social support, poor citizen
participation in volunteering and decision
making, poor sense of community, and/or
excessive/insufficient attachment to place)
Community competence
(i.e., political instability, poor leadership
skills, low literacy and education levels,
lack of experience with similar hazards,
and/or lack of community partnerships and
support)
'5'&&66&5
151
0
Q16
Q17
1
2
3
4
Information and communication
(i.e., unreliable or excessive amounts of
extraneous information, public opinion
ignored, lack of trusted media
spokesperson, ineffective risk messaging,
and/or poor communication infrastructure
capabilities)
Other infrastructure
(i.e., high degree of interdependence
among critical systems, lack of redundant
or backup systems, and/or weak or blocked
distribution networks)
Q18 In your estimation, which of the following factors contributed most to the level of complexity in the
scenario? Please check one box and explain your choice.
† Impact
† Uncertainty
† Vulnerability
† All of the above (factors influenced the assessment equally)
Please explain why:
Please rate the overall level of complexity of the scenario on a scale of 0 (not at all complex) to 4 (very
complex).
0
Q19
1
2
3
4
Overall, I would describe the level of
complexity for the scenario as:
Please rate your level of agreement with the following statements on a scale of 0 (strongly disagree) to 4
(strongly agree).
0
Q20
Q21
Q22
152
1
2
3
The first part of the scenario is realistic /
plausible.
The second part of the scenario is realistic
/ plausible.
The timeline for the scenario is realistic /
plausible.
'5'&&66&5
4
M.9
Scenario Assessment Guide: Train Derailment Scenario
Monday 11:45 am
It is the first Monday in June, during a heat wave that has left Gapville hot and humid.
At 11:45 am a freight train derails only half a kilometre away from the National Rail
(NR) station. The NR radio from the station for help, and inform responders there was a
train derailment, and an explosion has occurred.
Within minutes, first responders are on the scene. Six chemical containment cars are
breached, sending a large yellowish haze of chemical up into the air. Paramedics wear
respirators and gloves as they scramble to get the seriously wounded and chemically
exposed to hospital.
Monday 1:00 pm
NR authorities have identified the chemical leak as Carbonyl Chloride, a phosgene
chiefly used in the production of polymers including polyurethanes, and polycarbonates.
The toxicity of a phosgene is mainly due to the HCl (hydrogen chloride) that is released
with its reaction with water. Even small amounts of water can trigger the toxic reaction,
such as clouds, or even entry into the lungs.
A fatal dose of phosgene eventually leads to shallow breathing and retching, rapid pulse,
an ashen face and the discharge of 2 liters of yellow liquid from the lungs each hour for
the 48 hours of the drowning spasms that occur following exposure.
The protocol for an HCl leak is immediate removal from the area, and protection for
those downwind of the chemical. Equally important, those who have been in direct
contact with the chemical should not be taken to hospital, as they carry contaminants on
their clothing and skin. The hospital is immediately notified of the nature of the
chemical contamination. The hospital immediately performs decontamination of all the
train derailment victims and health care workers who came in contact with these
patients.
Monday 4:00 pm
Within a couple of hours, the chemical cloud has spread from the accident site at the
central north end of the city, through the downtown corridor to south central Gapville,
effectively splitting the city into two halves. Due to the high level of humidity the
normally heavy gas is able to react with low laying haze, and slowly drift through the
city.
Authorities are recommending evacuation for Gapvillians in the directly affected areas
due to the highly toxic nature of Carbonyl Chloride, and the forecasted continued hot
and humid weather conditions.
'5'&&66&5
153
Monday 5:00 pm
With the size and scope of the evacuation and decontamination, Gapville’s resources are
stretched to the limit. Police and the Fire department are still on scene at the site of the
derailment. Rescue and clean-up are incredibly difficult with the chemical cloud.
The McLaughlin Memorial Hospital is within the evacuation zone, and requires extra
help in moving patients to the High School and the Shopping Mall. EMS is working
with the municipal government, volunteer agencies and the hospital to evacuate to
safety.
The municipal authorities contact the Province, who immediately sends additional
resources to set up temporary decontamination sites as well as police officers to begin
enforcing an evacuation zone. The province also contacts the National military, who
announce they will arrive on Thursday morning. Some resources may be there as early
as Wednesday, including temporary shelters, decontamination supplies and personnel to
assist.
Monday 6:00 pm
The municipal government creates an evacuation message, however, the GapTV studios,
which contain the local broadcast tower, are also inside the evacuation zone, making
local radio and television transmissions impossible. A remote news van is used to
videotape the information, and send it to the national networks. The internet is rapidly
becoming the most used source for local information.
The local authorities seek to calm the people, explaining that contamination through the
air is far less toxic than direct contact. Equally they prescribe home decontamination
procedures.
“At temperatures above 7.6° C it is most likely that exposure will be solely to phosgene
vapor, a more harmless exposure than direct contact with the chemical. If there is no
evidence of exposure to liquid phosgene, decontamination simply requires removing the
victim from the contaminated area and removing outer clothing which should be bagged
and disposed of. Soapy water should be used to clean the skin using a rinse, towel off,
repeat process if liquid has been observed. Hair should be thoroughly cleaned using
soap and water, with care being taken to prevent wash water from contacting eyes.”
The local authorities also outline the evacuation zone, an 11 kilometer radius around the
toxic cloud. This takes up over one third of the city, a devastatingly large area, which
includes many of the city’s key infrastructures such as public transit and the bank.
Monday 9:30 pm
Evacuations are still moving slowly and mostly on foot. The toxic cloud ran very close
along the entire north-south GapTransit monorail line, and the Main Street terminal,
154
'5'&&66&5
effectively shutting down public transportation. Authorities need to wear respirators and
gown and gloves, as they go door to door asking residents to leave the affected area.
Tuesday 4:30 am
The evacuation zone is declared officially cleared of residents. Those most seriously
affected were carried on stretchers to decontamination and treatment centers. Exertion
increases the reaction within the lungs, so even ambulatory patients need to be carried if
possible. With the more urgent message being delivered from the National authorities,
many of those who could have performed a home decontamination arrive at the
treatment sites.
Tuesday 9:00 am
Almost a day later at the site of the train derailment, rescue workers have retrieved the
last of the victims. It is becoming obvious that the leaking chemical cars have
contaminated the river, which runs beside the tracks. The river provides the city’s local
water supply. It also is the watershed for a large portion of the province. The federal
government is contacted, and they send in environmental experts to assess the damage
being done to the city and province's water supply, and air quality.
Tuesday 9:15 am
National and provincial authorities prepare a press release of their own. A state of
emergency is declared for the province.
“For those responders in the area, complete chemical protective ensembles should be
worn to ensure protection. Self-contained breathing apparatus is required for all on-site
personnel. Volunteers should not enter the evacuation zone. All residents in the affected
zone are to evacuate uphill and upwind without moving through the agent cloud. A list
of shelters and decontamination / treatment sites are available. It is recommended you
seek treatment if you experience shortness of breath, or a burning of the eyes or mouth.
Water contaminated by HCl poses potential hazards, and clean up of this spill may be
extensive. Do not drink any tap water. Use bottled water only.”
INJECT – Task #1
Wednesday 10:00 am
Evacuation centers are full, and the military has arrived to help with the displaced
individuals, setting up shelters, medical stations, and distributing food and water
supplied through volunteer organizations throughout the province.
The federal environmental team conducted preliminary analysis, and are disheartened
with the results. With the contents of all six cars drained into the water supply, and the
toxic cloud hovering above the city’ downtown core, water, land and air are all affected.
With two days of exposure, there are many surfaces in the evacuation zone that are
contaminated, including public parks, businesses, schools, and homes.
'5'&&66&5
155
With the continuation of the heat wave, with a third day above 33° C, the situation is
dire. Air quality is poor, and heat exposure is also becoming an issue. Pregnant women,
the elderly and children are all at high-risk for adverse effects.
Wednesday 11:00 am
Wednesday morning the city is reeling from the upset of evacuation, and the uncertainty
surrounding the chemical leak. Carbonyl Chloride will leave residual effects, but the
degree of contamination is uncertain. The public has questions: How much Carbonyl
Chloride entered the River, the city's water supply? How much is deposited on surfaces?
Are our children safe? Have we already been contaminated? When can we go home?
Equally there are elderly and those with mobility restrictions that have been unable to
evacuate, and finding and helping these citizens is becoming the priority, as over 200
people have already fallen seriously ill, and there appears to be over 15 first responders
with potentially fatal exposure.
The internet chatter on all social networking sites contains both accurate and inaccurate
information. Many people are upset at the dispersion, with many family members stuck
on either side of the toxic cloud. Rumours regarding the toxic nature of the tap water
stop people from using it to decontaminate themselves as they believe this increases the
risk of exposure through surface contact.
Wednesday 12:00 pm
Parents are increasingly concerned about the latent effects of the phosgene on surfaces.
Equally, with many of the schools closed, there are significant interruptions for
education and workplace productivity. It has been reported that over 55 hospital patients
have passed away, due to either exposure to the chemical exposure, or through the
evacuation process. There are now 86 confirmed fatalities and over 450 seriously ill.
Thursday 7:30 pm
Many of the first responders and health care workers are becoming ill after prolonged
exposure to contaminated patients. Police, fire and EMS have been working at maximum
capacity for three days, and many are falling ill with respiratory difficulties.
Friday 8:00 am
Overnight the temperature broke, with a violent thunderstorm, high winds and heavy
rains. After four days apart from loved ones, people are getting anxious to return to
“normal”. Despite the continued enforcement of the evacuation zone, many people are
surreptitiously entering their homes and businesses, to collect personal belongings. Some
are crossing the evacuation zone to reach loved ones on the other side of the chemical
cloud. A number of people have been arrested in order to maintain the evacuation order;
this creates a serious public protest about heavy handed police authority.
156
'5'&&66&5
Friday 12:00 pm
Authorities are quick to release an announcement regarding the change in weather
conditions and how that impacts the evacuation zone. People are asked to stay out for
another two days. Compliance is becoming an issue. Internet chatter displays a public
mistrust in the authorities, and a frustration at being displaced for so long. Many who
have successfully entered or crossed the toxic area are reporting no ill effects, which is
causing more and more people to attempt to enter the evacuation zone.
Task #2
'5'&&66&5
157
M.10
Scenario Assessment Guide: Scenario Assessment
Questionnaire – Train Derailment Scenario
This form is designed to capture a reader’s understanding of what contributes to the complexity of a given
scenario. Potential contributions to complexity for a scenario are described below in terms of impact,
uncertainty and vulnerability, with specific challenges or areas noted for each. After reading the scenario,
the reviewer is asked to rate the degree to which each challenge was evident in the scenario. The results
will be compared with data collected from other reviewers to select a suitably complex scenario for use in a
simulation exercise.
Q1 Overall, which of the following descriptors best describes the scenario?
† Simple
† Complicated
† Complex
Please rate the extent to which each of the following challenges is evident in the scenario. Use the scale
ranging from 0 (not at all evident) to 4 (strongly evident).
0
1
2
3
Impact
Q2
Q3
Q4
Q5
Q6
Impact severity
(i.e., number of fatalities /casualties,
amount of physical damage/ psychological
trauma, degree of social disruption/multiple
events)
Impact scope
(i.e., breadth of impact on people, services,
structures, societal functions )
Impact timing
(i.e., no threat/warning phase, sudden
onset, sustained duration, and/or recurring
event)
Media involvement
(i.e., extensive media coverage, lack of
access to official sources, inappropriate
use of social media tools)
Political processes
(i.e., multiple organizations involved,
multiple jurisdictions, and/or multiple levels
of government, public fear and/or mistrust
in officials)
Uncertainty
158
'5'&&66&5
4
0
Q7
Q8
Q9
Q10
Q11
Q12
1
2
3
4
Novelty of situation
(i.e., novel hazard, malicious cause, lack of
experience, lack of controllability, and/or
latent impacts)
Anticipation and planning
(i.e., lack of planning, rehearsal, and
sharing of plans, and/or lack of integration
of plans)
Data and information
(i.e., lack of access to information,
inaccurate information, lack of credible
information sources, excessive volume of
data, and/or fast speed of information flow)
New organizations and partners
(i.e., presence of non-traditional
organizations and partners, and/or
presence of emergent organizations or adhoc groups that self-organize to fill
perceived resource gaps)
Changing context
(i.e., big changes in situation context, fast
speed of change, actual/perceived
negative consequences of change, and/or
negative impacts of intervention (negative
secondary effects)
Flexibility of interpretive frameworks
(i.e., imposed/unilateral processes, lack of
common/shared processes, and/or lack of
creativity/improvisation)
Vulnerability
Q13
Q14
Q15
Economic development
(i.e., lack of resources, lack of economic
diversity, unequal distribution of wealth,
and/or economic instability)
Social capital
(i.e., large number of at-risk populations,
lack of social support, poor citizen
participation in volunteering and decision
making, poor sense of community, and/or
excessive/insufficient attachment to place)
Community competence
(i.e., political instability, poor leadership
skills, low literacy and education levels,
lack of experience with similar hazards,
and/or lack of community partnerships and
support)
'5'&&66&5
159
0
Q16
Q17
1
2
3
4
Information and communication
(i.e., unreliable or excessive amounts of
extraneous information, public opinion
ignored, lack of trusted media
spokesperson, ineffective risk messaging,
and/or poor communication infrastructure
capabilities)
Other infrastructure
(i.e., high degree of interdependence
among critical systems, lack of redundant
or backup systems, and/or weak or blocked
distribution networks)
Q18 In your estimation, which of the following factors contributed most to the level of complexity in the
scenario? Please check one box and explain your choice.
† Impact
† Uncertainty
† Vulnerability
† All of the above (factors influenced the assessment equally)
Please explain why:
Please rate the overall level of complexity of the scenario on a scale of 0 (not at all complex) to 4 (very
complex).
0
Q19
1
2
3
4
Overall, I would describe the level of
complexity for the scenario as:
Please rate your level of agreement with the following statements on a scale of 0 (strongly disagree) to 4
(strongly agree).
0
Q20
Q21
Q22
160
1
2
3
The first part of the scenario is realistic /
plausible.
The second part of the scenario is realistic
/ plausible.
The timeline for the scenario is realistic /
plausible.
'5'&&66&5
4
Annex N
Decision Quality Rating Instrument
Decision Quality Rating Questionnaire
Decision Quality Rating
Please rate the extent to which the decisions reflect the following items on a scale of 0 to 4 (0 – to a very small extent;
1 – to a small extent; 2 – somewhat; 3 – to a great extent; 4 – to a very great extent).
To a very
small extent
0
To a small
extent
1
Somewhat
2
To a great
extent
3
To a very
great
extent
4
Q1
The decisions reflect multiple
perspectives from within the pod
ż
ż
ż
ż
ż
Q2
The decisions reflect multiple
perspectives from outside the
pod
ż
ż
ż
ż
ż
Q3
The decisions reflect
innovative/creative
characteristics
ż
ż
ż
ż
ż
Q4
The decisions reflect a unified,
cohesive approach
ż
ż
ż
ż
ż
Q5
The decisions reflect shared
information
ż
ż
ż
ż
ż
Q6
The decisions reflect shared
resources
ż
ż
ż
ż
ż
Q7
The decisions reflect shared
activities
ż
ż
ż
ż
ż
Q8
The decisions reflect shared
power/authority
ż
ż
ż
ż
ż
Q9
The decisions have significant
operational gaps or challenges
ż
ż
ż
ż
ż
Q10
The decisions have significant
strategic gaps/challenges
ż
ż
ż
ż
ż
'5'&&66&5
161
Please rate your level of agreement with the following statements on a scale of 0 to 4 (0 – strongly disagree; 1 –
disagree; 2 – neutral; 3 – agree; 4 – strongly agree).
Strongly
disagree
0
Disagree
1
Neutral
2
Agree
3
Strongly
agree
4
Q11
The pod considered the impacts
of the event on multiple tiers (first
tier is the direct damages to
victims and infrastructure; second
tier is the impact on core
services; and third tier is the
impact of an event on trust, order
of civil society, and social fabric).
ż
ż
ż
ż
ż
Q12
The pod considered the impacts
that implementing solutions
would have on multiple tiers (first
tier is the direct damages to
victims and infrastructure; second
tier is the impact on core
services; and third tier is the
impact of an event on trust, order
of civil society, and social fabric).
ż
ż
ż
ż
ż
Q13
The pod considered the
psychosocial implications of the
event.
ż
ż
ż
ż
ż
Q14
The pod considered the
psychosocial implications that
implementing solutions would
have on the population.
ż
ż
ż
ż
ż
Q15
The pod considered general
resource constraints and
availability in the decision making
process.
ż
ż
ż
ż
ż
Q16
The pod considered whether or
not specialized resources would
be required to implement
solutions.
ż
ż
ż
ż
ż
Q17
The pod considered the time
requirements that solutions would
require to be implemented.
ż
ż
ż
ż
ż
Q18
Considering the time and
resources available, the pod’s
decisions were good overall.
ż
ż
ż
ż
ż
162
'5'&&66&5
Annex O
PODS Participant Workbook Materials
Briefing Information
During the course of this experiment a facilitator will be on hand. The facilitator will conduct a
debriefing following the experiment. Participation in this experiment is voluntary.
As a participant, you will be immersed in a fictional scenario, asked to complete two tasks, and
asked to complete a series of questionnaires. These questionnaires will ask questions about your
background in the field of emergency response, your experience working on the two tasks, and
on how you interpreted the events in the scenario.
While the events in the scenario are fictional, certain aspects of this experiment will be realistic.
Tasks will have time limitations, and the scenario will continue whether the task is completed or
not. Please ensure that you take the time to fill out all questionnaires.
Participant Profile Instructions
Your role within the experiment will be based upon your experience in emergency response.
During the course of the experiment, please consider yourself as having been relocated to
Gapville, where your job resembles your current occupation with respect to authority, power,
role and responsibilities. You will be performing the same duties that you are usually responsible
for. If you are retired, please imagine that you are occupying your former job, in the city of
Gapville, and that you have come out of retirement for the course of the experiment.
Gapville Profile
Located in the Province of Anticipate, in the Country of Reframe, between Ripple River and the
Lemyre Mountain Range is Gapville. The city is only 80 km from the Provincial Capital, and is
only 20km away from the national border to the south. Gapville is a growing city with a
population of 205,000 with all the built and social infrastructures found in a medium sized North
American city. For Example, the Gapville Dam and Hydro Electric station provides power to the
entire town and the GAPTransit Commission runs a Light Rail system and bus routes.
In recent years Gapville’s Hi-Tech sector has expanded, creating an influx of immigration. The
leading technology company, P-RAM Communications employs over one quarter of the
community. Tourism is another large industry, thanks in part to the creation of the P-RAM
Pavillion built in 2004. This corporate-sponsored center draws tourists from all across Anticipate
and the Baseball Stadium is home to the Gapville Hazards.
Municipal and Provincial services are located in the Anticipate Building, beside the courthouse.
The Gapville Police Force and the Fire Department each have two divisions across the city. The
'5'&&66&5
163
University of Gapville is a fully accredited and well respected institution, as is the McLaughlin
Memorial Hospital which houses an excellent pre and post-natal facility – Claire’s Hope.
This center of excellence draws patients from all across Reframe. Gapville’s original
entertainment district contains the renowned Corneil Museum of Natural History and the Old
Town Historical Theatre. Gapville has a multi-faith population and provides several churches of
different denominations along with a mosque and Buddhist temple. Social infrastructure
includes outreach centers, social clubs, and neighbourhood watch.
164
'5'&&66&5
Gapville in Depth
Please use this table for reference throughout the course of the experiment on an as needed
basis.
Total Population
Geography
• 205,000 residents
•
•
•
•
•
•
•
•
•
Utilities
•
•
•
•
Telecommunications
•
•
Transportation
•
•
•
'5'&&66&5
In the province of Anticipate
In the country of Reframe
80 km from the Provincial Capital
20km away from the national border to the south
Located in a valley, 600 feet below sea level
Ripple River is located to the North West
Lemyre Mountain range is located to the North East
North temperate zone:
o Forest fires are prevalent in the autumn
o Floods in the rainy spring season are an annual issue
o Ice storms in the winter time are common in valley
areas
The Ripple River Dam and Hydro electric station; buried power
lines throughout the city
Natural gas piped into Gapville from major urban center,
Primerton, 20 kilometres away
Ripple River provides water supply directly into the town
GAPtv Head office
Broadcast tower serves:
o The local media outlet, GAPtv
o The Provincial public broadcast service AnticipateTV
o 2 local radio stations
o Telephone
o Wireless Internet
o Cellular Telephone Tower
Freight and passenger railway, along the Reframe National
Rail Line
Gapville Domestic airport serves Reframe National and
provincial air carriers
The Airport also provides charter flights and pilot training
Interprovincial Expressway serves 150,000 cars and trucks
daily from Primerton to Response with 2 on and off ramps in
downtown core
GAPTransit (GT) Public Transit Main Street Terminal Central
Hub
o GT light rail / GT bus routes
165
Government
Security
Health
Financial
Social Services
Industry
Communities
Tourism
Retail Market
166
• Anticipate Provincial and Reframe Municipal Building
• Gapville Courthouse
• Post office
• Gapville Fire Department: 2 divisions, 2 others in the rural
areas
• Gapville Police Department: 2 divisions downtown
• Medium Security Prison
• McLaughlin Memorial Hospital:
o Claire’s Hope Pre and Post Natal Care Unit
• Center-town Drop-In Clinic
• Reframe National Bank Service branch
• 4 ATMs throughout town
• Johnson Estates Community Center:
o Child care facility
o Library
o Track and field facility
o Swimming pool
• South-End food bank and outreach center
• Forestry, processing and distribution of raw timber across the
country in Industrial Park
• Riding Dairy and Vegetable Farm, shipping locally and
provincially
• Johnson Estates golf and country club service
• Johnson Estates home construction industry
• Hi-tech Sector, largest employer: P-RAM Communications
• Johnson Estates
• Desmarais Heights
• Center-town
• Retirement living facility also runs a meal delivery service
• P-ram pavilion, corporate sponsored center includes:
o Theatre, including standing room for up to 25,000
o Indoor parking for 2100
o Kid’s Village
o Horse racing
• 4 Airport area hotels can accommodate over 3000 guests each
for a total of 12,000 visitor occupancy at any time
• Ripple River Shopping Mall
• Center-town markets & cafes
• Asian Market
• Little Italy
'5'&&66&5
•
•
•
•
•
Education
Sports and Recreation
Arts
Entertainment
and
Religious Services
Volunteer Services
Social Clubs
'5'&&66&5
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
2 Elementary schools
1 High School
University of Gapville
Pinsent Park
Baseball Stadium:
o Gapville Hazards Baseball Team draws 19,800 people
on game night
o Underground parking for 1500
Hiking / Skiing in the Lemyre Mountain Range
Old Towne Movie Theatre
Corneil Museum of Natural History
Sociale Nightclub
Anticipate Art Gallery
1 Catholic Cathedral
1 Protestant Church
1 Buddhist Temple
1 Muslim Mosque
Meals with Wheels
Neighbourhood Watch
Helping Hand Immigration Center run out of the Mosque
providing:
o Translation services
o Employment help
Johnson Estates Private Country Club
Transportation Worker’s Club
Boy and Girls Club of Reframe
Italian Club
Gapville Juniors AAA Baseball League
Tai Chi Society, run out of the Temple, and using Pinsent Park
167
Situation Report #1
Incident Number: RD001-11
Incident: Radiological Bomb at the SSMD Warehouse
Date: Information valid as of Tuesday, 15th October, 17:50 hours (EST)
Description of current incident: At around 15:50, Monday October 14th, a package exploded in
the loading bay at the Sort & Send Mail Delivery (SSMD) warehouse. The explosion was from a
dirty bomb, originally intended for the GAPTV television station, and it has now spread several
ounces of radioactive Cesium 137 dust in the southern industrial sector of the city. The
contaminated areas include the South End LRT station and the International Borders and Customs
warehouse. The source of this attack and the motives behind it are unknown at this time.
Source(s) of reporting: Original source of reporting from responders on site, and GapTV
Broadcasts.
Current actions in response: Provincial HazMat teams were first on site after the local
responders, and are conducting investigations at this moment, with the help of municipal
responders. Public Safety and Emergency Preparedness is the primary federal department for
this incident, including public communications until otherwise advised.
Federal Police, the Security Intelligence Services and the National CBRN team are clearing the
contamination, tracking down all the potential victims of radiation exposure, and evacuating areas
determined to be in a “hot zone”. The Department of Defence, the Military and the National
Health Agency are all working on-site at the EOC.
Assessment/Analysis:
Seventeen people are wounded, and the six closest to the explosion are critically ill. Some of the
victims of the flying glass and debris show signs of mild to moderate radiation exposure.
Apparently the radiation sickness is worse if the victim has open wounds. There is also
considerable concern around pregnant women and children, as they are the most seriously
affected by the long term effects of radiation.
The 6 critically ill warehouse workers were taken to the McLaughlin Memorial Hospital by
ambulance.
These causalities contaminated the Hospital’s emergency room, and some of the patients and
health care workers who were in the ER at the time.
The Hospital has called a code brown for hazardous materials. The Hospital also has activated a
Memorandum of Understanding with the local pulp and paper mills to use their chemical
protection equipment, and self contained breathing apparati.
After the 4pm shift change at the Hospital, some contaminated health care workers went out in
public unknowingly spreading radiation carried on their skin and clothing. The contaminated
ambulances also went back out on call before they could be notified of their radiation exposure.
Infantry along with military medics are assisting the overrun at McLaughlin Memorial Hospital
168
'5'&&66&5
with moving critical patients to auxiliary treatment centers at the walk-in clinic and the Golden
Acres assisted living home.
The international press arrived around 10pm Monday evening in Gapville at the same time as
representatives from NATO, the United Nations and the International Atomic Energy Agency.
Contamination travelled on the light rail train and went through the downtown core, the university
campus, Little Italy, some parts of the northern industrial park and the airport.
Provincial Parks and Recreation are now working with Police, GAPTransit, Airport Authorities
and the Transportation Safety Board to keep traffic flowing, and to assist in the shelter and
support for all those who have arrived, and for all those who have been displaced.
Transportation Safety Board has agents at the Gapville Domestic Airport. They report elevated
readings at the airport that could be related to either naturally occurring radon, or from possibly
more malicious sources. Security Intelligence Services have arrived at the airport and have begun
to shut down the terminal.
Additional notifications: All the buildings in a 2 kilometre radius of the explosion have been
evacuated. This includes the massive international Borders and Customs warehouse, as well as
the LRT South End station, and two high rises; one is an apartment building, the other an office
tower.
Due to the issues at the Customs warehouse, the Border Services Agency has been notified, as
well as the Department of International Affairs.
The Transportation Safety Board and Airport Authority are working with the Security
Intelligence Service right now to evacuate the terminal and are redirecting commercial air traffic
to the provincial capital of Primerton.
Municipal transit services have also been halted for now, until we can track down all the
potentially exposed victims, and all areas of the city that have unacceptable levels of radiation.
90 prisoners at the Medium Security penitentiary who were outside in the exercise yard at the
time of the explosion may have been exposed to radiation as well. Correctional Services have
been notified.
Issued by: Gapville Unified Coordination Centre
Email/courriel: GOC @Gapville.CA Tel.: (100) 555-7000 Fax/Télécopieur: (100) 555-0999
'5'&&66&5
169
Public Health Agency Press Release Inject
PRESS RELEASE
One Thousand, Three Hundred and Twenty (1320) people have arrived at the examination centers
set up in the hospital parking lot. The six (6) Sort & Send warehouse employees who were nearest
to the explosion are all critically ill. The paramedics and the initial ER staff who treated the
warehouse employees were also exposed to moderate levels of radiation and have fallen ill.
The seven (7) GapTransit victims are in stable condition, as are the customs warehouse workers,
and the unprotected fire fighters, EMS and police officers originally at the scene. Their symptoms
may not appear for another day or so, but their long-term cancer risks are high.
Eighty-two (82) other people have been hospitalized and quarantined for treatment of sub-lethal
but serious effects of radiation exposure; another one-hundred and sixty five (165) are following
out-patient treatments. Many of the workers who left the site after the explosion have been
tracked down or have arrived at the hospital.
Thirty percent (30%) of hospital staff is working overtime, dealing with the influx of patients,
coupled with the loss of employees who have been exposed to radiation. There have been seven
(7) confirmed radiation exposure cases, amongst the health care workers.
Family members for some of the responders and workers have been found to have varying levels
of exposure as a result of the contaminated clothing brought home. Potentially exposed children
and pregnant women are now the focus of intensive surveillance and monitoring.
The hospital officially request support from the military’s engineers and medical unit to set up an
emergency field hospital. Additionally, the hospital requests that Military Medics take care of the
prisoners who were exposed, as they are currently at the end of their resources at the emergency
treatment center. NGO’s and Volunteer agencies are ready to offer aid, but do not have the
appropriate protective gear to access the hot zones. More resources will need to be reallocated to
the management of volunteers, for their own safety. Memos of Understanding (MOU’s) have
been activated with the Pulp and Paper mills, for their chemical protective gear. These are being
dispersed to the responders. The Gapville Red Cross have sent workers in droplet protection
suits to the quarantine centers, to donate clothing to those who have to submit theirs for
incineration, and to usher the public through triage at the field hospital.
170
'5'&&66&5
Situation Report #2
Incident Number: RD001-11
Incident: Radiological Bomb at the SSMD Warehouse
Date: Information valid as of Wednesday, 16th October, 18:00 hours (EST)
Description of current incident: At around 15:50 pm, Monday the 14th of October, a package
exploded in the loading bay at the Sort & Send Mail Delivery (SSMD) warehouse. The explosion
was from a dirty bomb, originally intended for the GAPTV television station, and it spread
several ounces of radioactive Cesium 137 dust in the southern industrial sector of the city.
The location where the bomb was produced has been identified and additional areas in the city
have been exposed to contamination. Evacuation and containment activities are being taken by
police assisted by military personnel. Large numbers of people have left the city while the
movement of the majority of the population has been restricted to ensure public safety as every
attempt is being made to determine if there are additional exposures.
Source(s) of reporting: Original source of reporting from Responders on Site, and GapTV
Broadcasts.
Current actions in response: The federal presence and the military arrived following the
implementation of the National Emergency Plan following a request by the province and the
municipality. Since that time, additional national and international resources have been arriving
across the city.
The priority continues to be concentrating on public safety and containment of sites. Local police
with the military assisting them in providing security at the prison on the exterior as well as at the
perimeter of various contaminated sites.
The military will continue to assist the hazmat teams and the forensic investigators in doing a city
wide assessment to detect any additional sites and if discovered they will provide additional
security to prevent people from further exposure. The current taskings for all national groups will
be centred on security and providing humanitarian assistance under the direction of local
authorities.
Assessment/Analysis:
SIS agents identified those responsible for the attack and where they assembled the bomb. The
perpetrators used a suburban house in Gapville’s upscale neighbourhood, the Johnson’s Estates.
Radiation levels there are moderate to high.
There are at least 20 houses that will need to be decontaminated or destroyed. The Fire
Department is working with HazMat and the CBRN Team right now, evacuating the Johnson’s
Estates. They are conducting a door to door canvas in the affected areas to ensure residents have
evacuated.
There is concern that the perpetrators possibly contaminated themselves during the bomb
assembly, and may be spreading the contamination as they travel.
'5'&&66&5
171
This would explain the unusually high radiation readings found at the Gapville airport.
The south-eastern section of the city is still closed off, with the south end highway exit shut
down. Some bus routes are still running, but the Light Rail is still shut down. Police are asking
that citizens do not cross the hot zone barriers
The municipal, provincial and federal teams have taken a variety of actions intended to restore
public safety and confidence. Overnight, the fire stations that went to the fire have been closed
and their equipment and vehicles are being decontaminated. The ambulances, the hospital
emergency room and the rooms where the patients were handled are also being decontaminated.
The city has set up a social media site, with Facebook and Twitter feeds, as well as the Ministry
of Resources radiation plume maps.
The Red Cross and other volunteers groups have set up temporary schools and daycares for all the
children who cannot attend classes today at the Johnson Public School.
There is a concern that residents and business owners are attempting to return to their properties
to protect them as rumours and media reports lead them to believe there has been looting in the
city. This is placing additional strain on those who are trying to maintain the security and safety
perimeters.
Additional notifications:
The Department of Foreign Affairs is also working with investigators over the international
contamination issues.
Long term environmental impact assessments are being requested.
Issued by: Gapville Unified Command Centre
Email/courriel: GOC @Gapville.CA Tel.: (100) 555-7000 Fax/Télécopieur: (100) 555-0999
172
'5'&&66&5
Facebook Posts Inject
'5'&&66&5
173
174
'5'&&66&5
Twitter Feed Inject
'5'&&66&5
175
Debriefing
Thank you for taking part in the scenario. Please join the other pod members for a
debriefing session in the plenary room. Feel free to ask the facilitator any questions that you
might have. Please hand in your binder and questionnaires to a staff member in the plenary
room.
During the debriefing session you will be asked to fill out an additional questionnaire about
how you interpreted the events in the scenario.
If you would like to discuss the scenario further, please contact us at GAP-Santé.
Louise Lemyre
Professor, School of Psychology
Faculty of Social Sciences
University of Ottawa
Tel: 613-562-5800 (ext 1196)
llemyre@uottawa.ca
Gap-Santé
Tel: (613) 562-5800, ext. 2321
Fax: (613) 562-5350
gapsante@uottawa.ca
176
'5'&&66&5
Annex P
P.1
Inter-GAP In Vivo Session Materials
Briefing Slide Deck
'5'&&66&5
177
P.2
Controller Script
CONTROLLER OPEN
Time & Slide
07:30 –
09:00
Activity
Setup
Duties
•
•
•
•
TECHNICAL:
Plug in headset
Start Nefsis Conference
Ensure all sound and video is functioning
properly
• REMEMBER*
• From now on, control mute through the
headset, not the software.
•
•
•
•
•
•
•
•
•
MEDIA:
Pre-load all media in THIS order:
PODS Flow 2feb2011. ppt
PODS Scenario Part 1.wmv
PODS Scenario Part 2.wmv
PODS Scenario Open Comm1.wmv
PODS Scenario Part 3.wmv
PODS Scenario Youtube.wmv
PODS Scenario Open Comm2.wmv
• POWER POINT:
• Begin with Flow Slide #1 Waiting on
Screen.... Ensure the Annotation function is
deselected.
• LAYOUT:
• Select Follow my Layout
• Select Style 1
09:00
Welcome
• TECHNICAL:
• Begin Nefsis Recording, GO TO: top left
icon; record conference;
• Use .avi codec and save to desktop.
• REMEMBER*
• Record Actual Desktop. Enable audio for
participant introduction.
• VOICE OVER:
• “Good Morning PODS, can you please
introduce yourselves to the camera, with
178
'5'&&66&5
your name and participant number, starting
with you POD one….”
Briefing
• TECHNICAL:
• Mute Controller Headset
• GO TO: Audio; Mute All
• POWER POINT:
• Move to Flow Slide #3 – leave on screen
for 30 seconds.
• VOICE OVER:
• Before the video: “Feel free to use the
color map of Gapville for reference, and
you have a black and white copy provided
in your binder, that you may draw and
write on as well.”
• MEDIA:
• GO TO: Layout; Share Only; Full Screen
• Inside the MEDIA PLAYER, select “Fit in
View” and turn down the media player
volume to about 1/3
• Play the Gapville Intro
09:10
Deliver
Scenario
(1st Half)
POWER POINT:
• Advance to Flow Slide #4
• Keep on screen for 30 seconds.
• MEDIA:
• Play VIDEO: PODS Scenario Part 1
• Play VIDEO: PODS Scenario Part 2
09:20 –
09:40
Begin Task
1
• TECHNICAL:
• RIGHT CLICK on all the POD rooms in
the user list. GO TO: Audio Settings
• Adjust POD Speakers to 0
• Ensure POD Microphone is still set at 90
• POWER POINT:
• Advance to Flow Slide #5, keep on screen.
• LAYOUT:
• GO TO: Share; Chat; Chat with all users
'5'&&66&5
179
• GO TO: Layout; Style 1; Fullscreen
• VOICE OVER:
• “I want to remind Participants of the twoway radios and highlight the chat function
on the right side of the screen…”
• Enter “Hello” in the chat box as an
example.
• Allow Participants 20 Minutes to complete
the task. Inform participants of their time
remaining, at 10 minutes, and 5 minutes.
09:40 –
10:15
Begin EOC
Conference
• POWER POINT:
• Advance to Flow Slide #6
• MEDIA:
• Play VIDEO: PODS Sit Rep Open Comm1
• LAYOUT:
• Save Chat file to Desktop as chat_date.rtf
• GO TO: Layout; Video Only; Fullscreen
• TECHNICAL:
• Unmute All Audio
VOICE OVER:
• “You should now be able to hear each
other. You have 35 minutes to complete the
task.”
• Remind participants of time remaining, at
20, 10 and 5 minutes.
10:15 –
10:30
Task 1
Questionna
ire
• LAYOUT:
• Exit Fullscreen to access layout panel.
• GO TO: Layout; Style 1; Fullscreen
• POWER POINT:
• Advance to Flow Slide #7
• VOICE OVER:
• “Once you have finished your
questionnaires we will have a short break.
The experiment will resume at (10:50am)”
180
'5'&&66&5
10:30 –
10:50
Break
10:50 –
11:00
Deliver
Scenario
(2nd Half)
• POWER POINT:
• Advance to Flow Slide #8
• POWER POINT:
• Advance to Flow Slide #9 – leave on
screen for 30 seconds.
• MEDIA:
• Play VIDEO: PODS Scenario Part 3
• Play VIDEO: PODS Scenario Part
Youtube
11:00 –
11:20
Begin Task
2
• TECHNICAL:
• Adjust all POD Speakers to 0
• Ensure POD Microphone set at 90
• POWER POINT:
• Advance to Flow Slide #10, keep on
screen.
•
•
•
•
LAYOUT:
Exit Fullscreen to access control panel.
GO TO: Layout; Style 1; Fullscreen
GO TO: Share; Chat; Chat with all users
• VOICE OVER:
• “Hello PODS, again I’d like to remind
participants of the two-way radios and the
chat function on the right of the screen.”
• Allow Participants 20 Minutes to complete
the task. Inform participants of their time
remaining, at 10 minutes, and 5 minutes.
'5'&&66&5
181
11:20 –
11:55
Begin EOC
Conference
• MEDIA:
• Play VIDEO: PODS Sit Rep Open Comm
2
• LAYOUT:
• Exit fullscreen to access control panel.
• Save Chat file to Desktop as
chat_task2_date.rtf
• End Chat Function
• GO TO: Layout; Share Only; Fullscreen
• POWER POINT:
• Advance to Flow Slide #11
• VOICE OVER:
• “You should now be able to hear each
other. You have 35 minutes to complete the
task.”
• Remind participants of time remaining, at
20, 10 and 5 minutes.
• LAYOUT:
• Exit fullscreen to access control panel.
• GO TO: Layout; Video Only; Fullscreen
11:55 –
12:10
Task 2
Questionna
ire
• POWER POINT:
• Advance to Flow Slide #12
• VOICE OVER:
• “Once you have finished your
questionnaires we will have a debrief and
lunch in the meeting room, (3120), where
we had our initial briefing. Please bring
your completed worksheets, questionnaires
and binder back to the room with you.”
12:10 –
12:15
Experiment
ends
• POWER POINT:
• Advance to Flow Slide #13
• TECHNICAL:
• Stop the Nefsis recording, and do
NOTHING while it renders and saves onto
182
'5'&&66&5
the desktop.
• Once all participants have left their pods
GO TO: Icon in top left, and End
Conference.
'5'&&66&5
183
CONTROLLER CLOSED
Time & Slide
07:30 –
09:00
Activity
Setup
Duties
•
•
•
•
TECHNICAL:
Plug in headset
Start Nefsis Conference
Ensure all sound and video is functioning
properly
• REMEMBER*
• From now on, control mute through the
headset, not the software.
•
•
•
•
•
•
•
•
•
MEDIA:
Pre-load all media in THIS order:
PODS Flow 2feb2011. ppt
PODS Scenario Part 1.wmv
PODS Scenario Part 2.wmv
PODS Scenario Open Comm1.wmv
PODS Scenario Part 3.wmv
PODS Scenario Youtube.wmv
PODS Scenario Open Comm2.wmv
• POWER POINT:
• Begin with Flow Slide #1 Waiting on
Screen.... Ensure the Annotation function is
deselected.
• LAYOUT:
• Select Follow my Layout
• Select Style 1
09:00
Welcome
• POWER POINT:
• Go to Flow Slide #2
• VOICE OVER:
• Ask Participants to Introduce themselves on
camera, with Participant ID Number
Briefing
• POWER POINT:
• Move to Flow Slide #3 – leave on screen for
30 seconds.
• VOICE OVER:
184
'5'&&66&5
• Before the video: “Feel free to use the color
map of Gapville for reference, and you have
a black and white copy provided in your
binder, that you may draw and write on as
well.”
• MEDIA:
• Inside the MEDIA PLAYER, select “Fit in
View”. Turn down the media player volume
to about 1/3
• Play the Gapville Intro video AFTER
allowing 30 seconds of screen time for Slide
#3
09:10
Deliver
Scenario
(1st Half)
• POWER POINT:
• Advance to Flow Slide #4
• MEDIA:
• Play VIDEO: PODS Scenario Part 1
• Play VIDEO: PODS Scenario Part 2
09:20 –
10:10
Begin Task
1
• POWER POINT:
• Advance to Flow Slide #5, keep on screen.
• VOICE OVER:
• Allow participants 50 minutes to complete
the task.
• Inform participants of their time remaining,
at 10 minutes, and 5 minutes.
10:10 –
10:25
Task 1
Questionna
ire
• POWER POINT:
• Advance to Flow Slide #6
• VOICE OVER:
• “Once you have finished your questionnaires
we will have a short break. The experiment
will resume at (10:50am)”
'5'&&66&5
185
10:25 –
10:50
Break
10:50 –
11:00
Deliver
Scenario
(2nd Half)
• POWER POINT:
• Advance to Flow Slide #7
• TECHNICAL:
• Wait for all participants to arrive back
• POWER POINT:
• Advance to Flow Slide #8
• MEDIA:
• Play VIDEO: PODS Scenario Part 3
• Play VIDEO: PODS Scenario Part Youtube
11:00 –
11:50
Begin Task
2
• POWER POINT:
• Advance to Flow Slide #9, keep on screen.
• VOICE OVER:
• Allow Participants 50 Minutes to complete
the task.
• Inform participants of their time remaining,
at 10 minutes, and 5 minutes.
11:50 –
12:10
Task 2
Questionna
ire
• POWER POINT:
• Advance to Flow Slide #10
• VOICE OVER:
• “Once you have finished your questionnaires
we will have a debrief and lunch in the
meeting room, (3120), where we had our
initial briefing. Please bring your completed
worksheets, questionnaires and binder back
to the room with you.
12:10 –
12:15
Experiment
ends
• POWER POINT:
• Advance to Flow Slide #11
• TECHNICAL:
• Stop the Nefsis recording, and do NOTHING
while it renders and saves onto the desktop.
186
'5'&&66&5
• Once all participants have left their pods GO
TO: Icon in top left, and End Conference.
'5'&&66&5
187
P.3
Session Administration Information Form
Session Administration Information Form
Date (dd/mm/yyyy):
Session #:
Session Type (check ż1 University of
one): Ottawa Students
ż2 Professional
Students
Task Type (check ż1 Coordination
one):
ż2 Collaboration
ż3 Senior Officials
Pod Status:
Pod Number One: ż0Not filled with
participants
ż1 Connected
ż2 Closed
Pod Number Two: ż0Not filled with
participants
ż1 Connected
ż2 Closed
188
# of Participants /
Pod:
Pod Type:
ż0None
ż0 Not Applicable
ż1 One
ż1 Homogeneous
ż2 Two
ż2 Mixed
ż3 Three
ż0None
ż0 Not Applicable
ż1 One
ż1 Homogeneous
ż2 Two
ż2 Mixed
ż3 Three
'5'&&66&5
Pod Number Three: ż0Not filled with
participants
ż1 Connected
ż2 Closed
ż0None
ż0 Not Applicable
ż1 One
ż1 Homogeneous
ż2 Two
ż2 Mixed
ż3 Three
# of Participants /
Pod:
Pod Status:
Pod Number Four: ż0Not filled with
participants
ż1 Connected
ż2 Closed
ż0None
ż0 Not Applicable
ż1 One
ż1 Homogeneous
ż2 Two
ż2 Mixed
ż3 Three
Scenario:
Location:
Observer (initials):
Facilitator:
Start Time / End Time
(hh:mm):
'5'&&66&5
:
Pod Type:
/
:
Session duration
(hh:mm):
:
189
P.4
Technical and Process Issues Form
Technical and Process Issues Form
Date
(dd/mm/yyyy):
Session #:
Audio ż1 Yes
Simulation
Delivery ż2 No
Problem(s):
Nature of the problem:
Video ż1 Yes
Simulation
Delivery ż2 No
Problems(s):
Nature of the problem:
Participant ż1 Yes
Audio
Recording ż2 No
Problem(s):
Nature of the problem:
Participant ż1 Yes
Video
Recording ż2 No
Problems(s):
Nature of the problem:
Inter-pod ż1 Yes
Communication
Problem(s) due ż2 No
to Technology:
Nature of the problem:
190
'5'&&66&5
Conferencing ż1 Yes
Software
Problem(s): ż2 No
Nature of the problem:
Internet ż1 Yes
Connection
Failure(s): ż2 No
Nature of the problem:
Other Technical ż1 Yes
Issue(s):
ż2 No
Nature of the problem:
Participant(s) ż1 Yes
Absent:
ż2 No
Nature of the problem:
Participant(s) ż1 Yes
Late:
ż2 No
Nature of the problem:
Participant(s) ż1 Yes
Left Early:
ż2 No
Nature of the problem:
Consent ż1 Yes
Refusal(s):
ż2 No
Nature of the problem:
Delayed Start ż1 Yes
Time:
ż2 No
Nature of the problem:
Session ż1 Yes
Interruption(s):
ż2 No
Nature of the problem:
'5'&&66&5
191
Participant(s) ż1 Yes
Confused by
Directions: ż2 No
Nature of the problem:
Participant(s) ż1 Yes
Confused by
Software: ż2 No
Nature of the problem:
Other Process ż1 Yes
Issue(s):
ż2 No
Nature of the problem:
192
'5'&&66&5
Annex Q
Q.1
Recruitment Documents
Student Recruitment Pool Posting
'5'&&66&5
193
Q.2
Participation Invitation Letter
'HDULQVHUWQDPH!
,DPLQYLWLQJ\RXWRSDUWLFLSDWHLQDVWXG\DWWKH8QLYHUVLW\RI2WWDZD2XUUHVHDUFKWHDPKDV
UHFHLYHG\RXUQDPHIURPLQVHUWUHIHUHQFH!3OHDVHVHHEHORZIRUDEULHIGHVFULSWLRQRIRXU
VWXG\:HEHOLHYHWKDWSDUWLFLSDQWVZLOOILQGLWDQLQWHUHVWLQJH[SHULHQFH
3UREOHPVROYLQJDQG2UJDQL]DWLRQDO'HFLVLRQPDNLQJ6LPXODWLRQ32'63URMHFW
Needed:3DUWLFLSDQWVIRUDQLQQRYDWLYHVLPXODWLRQH[HUFLVHRQSUREOHPVROYLQJDQGGHFLVLRQ
PDNLQJZLWKLQWKHFRQWH[WRIDQH[WUHPHHYHQW7KHUHVHDUFKLVEHLQJFRQGXFWHGE\WKH8QLYHUVLW\
RI2WWDZDDQGIXQGHGE\'HIHQFH5HVHDUFKDQG'HYHORSPHQW&DQDGD'5'&
Purpose of the study:7RH[DPLQHKRZRIILFLDOVZRUNWRJHWKHUGXULQJHPHUJHQFLHVZKLFKUHTXLUH
MRLQWUHVSRQVHVIURPERWKWUDGLWLRQDOILUVWUHVSRQGHUVSROLFHILUHDQG(06SXEOLFKHDOWKDQG
QRQWUDGLWLRQDORQHVVRFLDODQGFRPPXQLW\VHUYLFHVQRQJRYHUQPHQWDORUJDQL]DWLRQVRU
YROXQWHHUJURXSVDVZHOODVIURPWKHPLOLWDU\RURWKHUJRYHUQPHQWDJHQFLHV7KHRYHUDOOREMHFWLYH
RIWKHUHVHDUFKLVWRLGHQWLI\ZD\VWRLPSURYHLQWHURSHUDELOLW\EHWZHHQRUJDQL]DWLRQVGXULQJDODUJH
VFDOHHPHUJHQF\HYHQWRUWKUHDW
Who:6HQLRUOHYHOHPHUJHQF\PDQDJHPHQWSUDFWLWLRQHUVZLWKDWOHDVWRQH\HDURIH[SHULHQFHZLWK
KLJKOHYHOGHFLVLRQPDNLQJLQPDQDJLQJDQGUHVSRQGLQJWRHPHUJHQFLHV
When6HVVLRQGDWHVLQFOXGH
UG
)HEUXDU\ :HGQHVGD\
)HEUXDU\WK±7KXUVGD\
)HEUXDU\WK±)ULGD\
)HEUXDU\WK±0RQGD\
0DUFKVW±7XHVGD\
0DUFKQG±:HGQHVGD\
0DUFKUG±7KXUVGD\
0DUFKUG±:HGQHVGD\
6HVVLRQVZLOOEHKHOGLQWKHPRUQLQJVEHWZHHQ$OLJKWOXQFKZLOOEHSURYLGHG
Where:'HVPDUDLV+DOO5RRP8QLYHUVLW\RI2WWDZD/DXULHU(DVW.11
What:'XULQJVHVVLRQVSDUWLFLSDQWVZLOOEHDVVLJQHGWRJURXSVRIWKUHHSDUWLFLSDQWVFDOOHGD
SRG7KHVHSRGVZLOOEHLPPHUVHGLQDWDEOHWRSH[HUFLVHZKLFKLVGULYHQE\D&%51HVFHQDULR
DQGVLWXDWHGLQWKHILFWLRQDOPLGVL]HGFLW\RI³*$39LOOH´7KHSDUWLFLSDQWVZLOOZRUNLQVPDOOJURXSV
RQDVHULHVRIWDVNVWRDGGUHVVSDUWLFXODUDVSHFWVRIWKHHPHUJHQF\,QVRPHFDVHVSDUWLFLSDQWV
ZLOOEHFRPPXQLFDWLQJZLWKWKHRWKHUSRGVZLWKWKHDLGRIYLGHRFRQIHUHQFLQJVRIWZDUH$OODVSHFWV
RISDUWLFLSDWLRQDUHYROXQWDU\<RXZLOOEHDVNHGWRUHDGWKURXJKDQGVLJQDFRQVHQWIRUP
LQGLFDWLQJLQIRUPHGFRQVHQWSULRUWRSDUWLFLSDWLRQLQWKHVHVVLRQ
3OHDVHFRQWDFW+LODU\.LWFKHQHUDWJDSVDQWH#XRWWDZDFDRU[WRLQGLFDWH
\RXULQWHUHVWLQSDUWLFLSDWLQJ:HZLOOFRQWDFW\RXWRFRQILUPDUUDQJHPHQWV
6LQFHUHO\
LQVHUWQDPHVLJQDWXUH!
194
'5'&&66&5
Q.3
Phone Confirmation Script
Phone Confirmation Script
“Hello,
My name is [first name of caller], I am calling from GAP-Santé regarding your participation in
our Problem-solving and Organizational Decision-making Simulation (PODS) Project.
In order to meet the criteria for participation I need to ask you five questions.
1. Have you had previous experience in both strategic and operational decision making during
at least one major event?
2. Have you been in a management level, decision making role at your organization for at least
12 months?
3. Do you have the authority to make decisions regarding the allocation of resources on behalf
of your organization?
4. Are you able and willing to participate in this study?
5. Are you able and willing to participate in English?”
[If ANY answer is NO:]
“We really appreciate you taking the time to answer these questions. Unfortunately we will be
unable to ask you to participate in this study, at this time.
Once again, thank you for your time.”
[If ALL answers are YES:]
“Great. I would like to confirm your participation, and select a date for your session. Which of
these dates would suit you best? We have openings on …”
[Refer to dates with open time slots and schedule a mutually agreeable timeslot for the
participant]
“The study is being conducted on campus at the University of Ottawa in the Demarais building.
The address of the building is 55 Laurier East. Please meet us on the third floor in room 3120 at
8:30 am.
A few days before your session, I will e-mail you to remind you of the appointment. If you have
to cancel, please contact us at gapsante@uottawa.ca or (613) 562-5800 Ext. 2321.
I look forward to meeting you on [date of session]. Thank you very much for helping us with our
research.”
'5'&&66&5
195
Q.4
Reminder E-mail
Dear <insert participant name>,
This is a reminder of your upcoming appointment to participate in the Problem-solving and
Organizational Decision-making Simulation (PODS) Project experiment on <insert date>.
Please come to room 3120 on the 3rd floor of the Desmarais Building located at the University of
Ottawa; there will be signs to help direct you. The session begins at 8:30 am and runs until 12:30
pm. A light lunch will be served following the session.
The Desmarais Building is located at 55 Laurier Avenue East (click here for a link to our location on
Google Maps). If you arriving by bus, please exit at Laurier Station. Directions to the Desmarais
Building by car can be found following this e-mail.
Underground parking is available in the Desmarais Building (at a cost of $4.00 an hour, prepaid).
In order to access the parking, you must take the 417E exit at Nicholas Street. Next, take your
first right after you go across the Laurier intersection. Alternate options for parking are available
in the Byward Market or at the Rideau Centre.
If you have any questions about the study, please do not hesitate to contact Hilary Kitchener by email at hkitc036@uottawa.ca or by phone at 613-562-5800 ext. 2321. If, for any reason, you need
to cancel your appointment please let us know as soon as possible.
Sincerely,
<insert name of sender>
Directions to Desmarais Building, University of Ottawa
Desmarais Building from the East
The main access to Ottawa from the East is via Highways 417 and 174.
Off Highway 417 West, take the Nicholas/Mann exit
Follow the Nicholas Street exit to Laurier Avenue. (Continue straight ahead, do not turn
on Laurier)
• Turn right (east) immediately after Laurier Avenue intersection to access the Desmarais
Building parking lot.
•
•
Desmarais Building from the West
The main access to Ottawa from the West is via Highways 417 and 7.
•
196
Off Highway 417 East, take the Nicholas/Lees exit.
'5'&&66&5
Follow Nicholas Street to the first set of lights (Laurier Avenue) (Continue straight
ahead, do not turn on Laurier)
• Turn right (east) immediately after Laurier Avenue intersection to access the Desmarais
Building parking lot.
•
Desmarais Building from the South
The main access to Ottawa from the South is via Highways 16, 31 or 416.
From Highway 16:
Follow Prince of Wales Drive to Carling Avenue
Turn right (east) on Carling Avenue
Follow Carling Avenue to Bronson Avenue
Turn left (north) on Bronson Avenue
Follow Bronson Avenue to Laurier Avenue
Turn right (east) on Laurier Avenue
Follow Laurier Avenue to Waller Street
Turn right (south) on Waller Street, and follow loop around the condominiums to
Nicholas Street
• Turn right (north) on Nicholas Street
• Turn right (east) immediately after Laurier Avenue intersection to access the Desmarais
Building parking lot.
•
•
•
•
•
•
•
•
From Highway 31:
Follow Bank Street to Isabella Street
Turn right (east) on Isabella Street
Off Isabella Street, take the Highway 417 East on-ramp
Off Highway 417 East, take the Nicholas/Lees exit.
Follow Nicholas Street to the first set of lights (Laurier Avenue) (Continue straight
ahead, do not turn on Laurier)
• Turn right (east) immediately after Laurier Avenue intersection to access the Desmarais
Building parking lot.
•
•
•
•
•
From Highway 416:
Take the Highway 417 East on-ramp
Off Highway 417 East, take the Nicholas/Lees exit.
Follow Nicholas Street to the first set of lights (Laurier Avenue) (Continue straight
ahead, do not turn on Laurier)
• Turn right (east) immediately after Laurier Avenue intersection to access the Desmarais
Building parking lot.
•
•
•
Desmarais Building from the North (Quebec)
The main access to Ottawa from the North is via Highways 5, 50 and 148 in Quebec.
'5'&&66&5
197
•
•
•
•
•
•
•
198
Follow the indications for Ottawa via Macdonald Cartier Bridge
Off the Macdonald Cartier Bridge, follow the indications for King Edward Avenue
Follow King Edward Avenue to Laurier Avenue
Turn right (west) on Laurier Avenue
Follow Laurier Avenue to the third set of lights (Nicholas Street)
Turn right (north) on Nicholas Street
Turn right (east) to access the Desmarais Building parking lot.
'5'&&66&5
Q.5
Thank You Letter (pilot testing)
<Insert date>
Dear <Ms./Mr. insert last name>,
On behalf of Gap-Santé, we would like to take this opportunity to thank you for your invaluable
input into the Problem-solving and Organizational Decision-making Simulation (PODS) Project
at the University of Ottawa, funded by Defence Research and Development Canada (DRDC), the
Natural Sciences and Engineering Research Council of Canada (NSERC), and the Social Sciences
and Humanities Research Council (SSHRC). With the help of participants such as yourself, this
project will contribute to a better understanding of inter-organizational relationships and shared
decision-making in the context of emergency events. .
Again, with my colleagues Wayne Corneil, Paul Boutette, Celine Pinsent, and our whole team of
graduate students and research assistants, we sincerely appreciate your contribution of time as
well as your thoughtful participation in the PODS project. We acknowledge your support for the
advancement of knowledge in emergency response and psychosocial preparedness for extreme
events in Canada.
Sincerely,
Louise Lemyre, Ph.D, MSRC / FRSC
École de psychologie / School of Psychology
Faculté des sciences sociales / Faculty of Social Sciences
Institut de santé des populations / Institute of Population Health
Université d'Ottawa / University of Ottawa
55 Laurier E (3215) Pavillon Desmarais Hall
Ottawa, ON, CANADA K1N 6N5
tel: +1 613-562-5800 (x1196)
fax: +1 613-562-5350
'5'&&66&5
199
www.gapsante.uottawa.ca
louise.lemyre@uottawa.ca
200
'5'&&66&5
Q.6
Thank You Letter (senior level participants)
<Insert date>
Dear <Ms./Mr. insert last name>,
On behalf of Gap-Santé, we would like to take this opportunity to thank you for your invaluable
input into the Problem-solving and Organizational Decision-making Simulation (PODS) Project
at the University of Ottawa. With the help of participants such as yourself, this project will
contribute to a better understanding of inter-organizational relationships and shared decisionmaking in the context of emergency events.
This project is funded by Defence Research and Development Canada (DRDC), the Natural
Sciences and Engineering Research Council of Canada (NSERC), and the Social Sciences and
Humanities Research Council (SSHRC). The overall objective of the research project is to
identify ways to improve collaboration between the Canadian Forces, emergency response
professionals, and other non-traditional partners during extreme events. To this end, as a senior
level actor with considerable experience pertaining to these types of events, your participation has
been indispensable to the PODS in vivo experiment. Results from the experiment will be used to
formulate recommendations for effective communication and shared decision making within the
context of the Incident Command Structure.
Again, with my colleagues Dr Wayne Corneil, Dr Celine Pinsent, Paul Boutette, and our whole
team of doctoral students and research assistants, we sincerely appreciate your contribution of
time as well as your thoughtful participation in the PODS project. We acknowledge your support
for the advancement of knowledge in emergency response and psychosocial preparedness for
extreme events in Canada.
Sincerely,
Louise Lemyre, Ph.D, MSRC / FRSC
École de psychologie / School of Psychology
'5'&&66&5
201
Faculté des sciences sociales / Faculty of Social Sciences
Institut de santé des populations / Institute of Population Health
Université d'Ottawa / University of Ottawa
55 Laurier E (3215) Pavillon Desmarais Hall
Ottawa, ON, CANADA K1N 6N5
tel: +1 613-562-5800 (x1196)
fax: +1 613-562-5350
www.gapsante.uottawa.ca
louise.lemyre@uottawa.ca
202
'5'&&66&5
Annex R
Full Pod Session Equipment
Requirements
Quantity
Item
Description
3
POD rooms
Rooms sufficiently large enough to comfortably
Rooms should be in close
hold three participants as well as a computer and all proximity to each other to
camera equipment. Each room will require a wired assist in troubleshooting
internet connection.
1
Control room
Room large enough for at least two administrators
and two computers. Ideally, it could also have
enough space for observers. Like the POD rooms, it
must also have a wired internet connection.
3
Web camera
Camera software may need to be installed onto the
computers prior to starting the experiment.
Logitech Web cam C260
3
USB
Conferencing
mics/speakers
Web cameras often are equipped with internal
microphones, but they create feedback. USB
conferencing combined microphone/speaker sets
solve this issue.
Phoenix PCS duet
conference phone
10
Computers
Any brand of PC
Four computers for open and closed Pods. These
computers should include a large monitor suitable
for sharing among several people. Computers
should also have a minimum of three USB slots as
they will be used for the microphones, web cameras
and printers. Computers must also be running
Windows XP or higher
Two computers for the control room. One the host
the open Pods and one to host the closed Pods.
Four computers for observers to watch the session
either locally or remotely.
3
Video camera
Because the PODS experiment lasts several hours
Sony DCRSX63 or higher
cameras should have internal hard drives suitable for
recording up to three hours consecutively.
3
Magic Arms
These products can be purchased through most
photography stores and will be used to install the
video cameras in each room. Important: please
ensure that camera mounts are included with each
magic arm.
2
Microphone/earp These will be used in the control room.
hone headset
Logitech USB Headset
H360
4
Walkie talkies
Cobra CXT85C GMRS
radios
'5'&&66&5
These will be used as an alternative communication
method between PODS
Recommendation
Manfrotto 143 Magic Arm
Complete
203
Quantity
Item
Description
Recommendation
4
Digital Voice
recorder
One recorder will be set up to record Walkie Talkie
communications.
The remaining three will be set up in each Pod to
record all Pod audio.
We opted to use an extra
laptop and recorded
through its internal
microphone.
16
Licences internet This will be the method for deploying information to NEFSIS web conferencing
web
the pods participants. It will also serve as a platform NEFSIS.COM
conferencing
for communication between PODS. Participants are
free to speak, chat or even email between PODS
204
'5'&&66&5
This page intentionally left blank.
'5'&&66&5
205
List of symbols/abbreviations/acronyms/initialisms
ANCOVA
Analysis of Covariance
ANOVA
Analysis of Variance
ATM
Automated Teller Machine
BBQ
Barbeque
2
C
Command and Control
CB
Citizens’ Band
CBRN
Chemical, Biological, Radiological, Nuclear
CBRNe
Chemical, Biological, Radiological, Nuclear, Explosive
CF
Canadian Forces
Co.
Company
CORA
Centre for Operational Research and Analysis
CRTI
Chemical, Biological, Radiological, Nuclear and Explosives Research and
Technology Initiative
CSS
Centre for Security Sciences
Demo
Demonstration
DMS
Demarais
DND
Department of National Defence
DRDC
Defence Research & Development Canada
DRDKIM
Director Research and Development Knowledge and Information
Management
EMO
Emergency Measures Organization
EMS
Emergency Medical Services
EOC
Emergency Operations Centre
ER
Emergency Room
FC
Forces canadiennes
Feds
Federal Government
G8
Group of Eight
G-20
Group of Twenty
GAP
Groupe d’Analyse Psychosociale
HazMat
Hazardous Materials
HCl
Hydrogen Chloride
206
'5'&&66&5
ICS
Incident Command System
ISPR
Psychology Integrated System of Participation in Research
GT
Gap Transit
ICU
Intensive Care Unit
ID
Identification
MANCOVA
Multivariate Analysis of Covariance
MANOVA
Multivariate Analysis of Variance
MOU
Memorandum of Understanding
MP3
MPEG-1 Audio Layer 3
NGO
Non-governmental Organization
NI
Non-Incident Command System
NIMS
National Incident Management System
No.
Number
Non-ICS
Non-Incident Command System
NSERC
Natural Sciences and Engineering Research Council of Canada
PC
Personal Computer
PODS
Problem-solving and Organizational Decision-making Simulation
PRiMer
Psychosocial Risk Manager
R&D
Research & Development
RCMP
Royal Canadian Mounted Police
SARS
Severe Acute Respiratory Syndrome
SDM
Shared Decision Making
SPSS
Statistical Package for the Social Sciences
SSHRC
Social Sciences and Humanities Research Council
SSMD
Sort & Send Mail Delivery
TA
Technical Authority
TBD
To be decided
TIF
Technology Innovation Fund
TCPS
Tri-Council Policy Statement
uOttawa
University of Ottawa
USB
Universal Serial Bus
'5'&&66&5
207
This page intentionally left blank.
208
'5'&&66&5
DOCUMENT CONTROL DATA
(Security classification of title, body of abstract and indexing annotation must be entered when the overall document is classified)
25,*,1$725(The name and address of the organization preparing the document.
Organizations for whom the document was prepared, e.g. Centre sponsoring a
contractor's report, or tasking agency, are entered in section 8.)
6(&85,7<&/$66,),&$7,21
(Overall security classification of the document
including special warning terms if applicable.)
'HIHQFH5'&DQDGD±&66
1HSHDQ6W
2WWDZD2QWDULR.$.
81&/$66,),('
121&21752//('*22'6
'0&$5(9,(:*&(&-81(
7,7/((The complete document title as indicated on the title page. Its classification should be indicated by the appropriate abbreviation (S, C or U)
in parentheses after the title.)
5HVHDUFK8VLQJ,Q9LYR6LPXODWLRQRI0HWD2UJDQL]DWLRQDO6KDUHG'HFLVLRQ0DNLQJ6'07DVN
±7HVWLQJWKH6KDUHG'HFLVLRQ0DNLQJ)UDPHZRUNLQ9LYR
$87+256(last name, followed by initials – ranks, titles, etc. not to be used)
/HP\UH/HWDO
'$7(2)38%/,&$7,21
(Month and year of publication of document.)
December 2011
D 122)3$*(6
E 122)5()6
(Total containing information,
(Total cited in document.)
including Annexes, Appendices,
etc.)
231
11
'(6&5,37,9(127(6(The category of the document, e.g. technical report, technical note or memorandum. If appropriate, enter the type of report,
e.g. interim, progress, summary, annual or final. Give the inclusive dates when a specific reporting period is covered.)
Contract Report
6321625,1*$&7,9,7<(The name of the department project office or laboratory sponsoring the research and development – include address.)
Defence R&D Canada – CSS
222 Nepean St.
Ottawa, Ontario K1A 0K2
D 352-(&725*5$1712(If appropriate, the applicable research
and development project or grant number under which the document
was written. Please specify whether project or grant.)
E &2175$&712(If appropriate, the applicable number under
which the document was written.)
W7714-083659/001/SV
D25,*,1$725
6'2&80(17180%(5(The official document
number by which the document is identified by the originating
activity. This number must be unique to this document.)
DRDC CSS CR 2011-32
E27+(5'2&80(1712V(Any other numbers which may be
assigned this document either by the originator or by the sponsor.)
'2&80(17$9$,/$%,/,7< (Any limitations on further dissemination of the document, other than those imposed by security classification.)
Unlimited
'2&80(17$11281&(0(17(Any limitation to the bibliographic announcement of this document. This will normally correspond to the
Document Availability (11). However, where further distribution (beyond the audience specified in (11) is possible, a wider announcement
audience may be selected.))
8QOLPLWHG
'5'&&66&5
209
$%675$&7(A brief and factual summary of the document. It may also appear elsewhere in the body of the document itself. It is highly desirable
that the abstract of classified documents be unclassified. Each paragraph of the abstract shall begin with an indication of the security classification
of the information in the paragraph (unless the document itself is unclassified) represented as (S), (C), (R), or (U). It is not necessary to include
here abstracts in both official languages unless the text is bilingual.)
This report documents the completion of Task 3 of the work stream “Research Using in Vivo
Simulation of Meta-Organizational Shared Decision Making (SDM)”, one component of the
Technology Innovation Fund (TIF) program on Meta-organizational Collaboration that has been
designed to assist in understanding challenges faced by the Canadian Forces (CF). The objective
of the stream is to conduct basic research into shared decision making through the analysis of
case studies, exercises and simulations. Task 3 involved the development and testing of the
shared decision making framework in vivo. The research at this stage is to demonstrate that the
model when implemented in vivo can produce improvements in problem solving processes and
outcomes such as better quality decisions, higher levels of satisfaction with problem solving
processes, better time-to-satisfaction ratio and more cohesive multi-organization groups.
Ce rapport traite de l’exécution de la Tâche 3 du projet intitulé : « Recherche par la simulation
in vivo sur la prise de décision partagée des méta-organisations », une des composantes du
programme du Fonds pour l’innovation technologique (FIT) relatif à la collaboration métaorganisationnelle, qui a été conçu afin d’améliorer la compréhension des défis auxquels font
face les Forces canadiennes (FC). L’objectif de ce volet particulier est de mener une recherche
de base sur le partage des décisions au moyen d’études de cas, d’exercices et de simulations. La
Tâche 3 consistait à élaborer le cadre de partage des décisions in vivo et d’en faire l’essai. À ce
stade, la recherche visait à démontrer que le modèle, lorsqu’il est mis en œuvre in vivo, peut
aider à améliorer les processus de résolution des problèmes et leurs résultats, notamment des
décisions de meilleure qualité, de hauts niveaux de satisfaction en ce qui touche les processus de
résolution des problèmes, un meilleur rapport temps-satisfaction et des groupes multiorganisationnels plus cohésifs.
.(<:25'6'(6&5,37256RU,'(17,),(56(Technically meaningful terms or short phrases that characterize a document and could be
helpful in cataloguing the document. They should be selected so that no security classification is required. Identifiers, such as equipment model
designation, trade name, military project code name, geographic location may also be included. If possible keywords should be selected from a
published thesaurus, e.g. Thesaurus of Engineering and Scientific Terms (TEST) and that thesaurus identified. If it is not possible to select
indexing terms which are Unclassified, the classification of each should be indicated as with the title.)
Inter-organizational; collaboration; cooperation; coordination; decision making; problem
solving; complex situations; TIF; meta-organization
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertising