Chronic Diseases and Injuries in Canada Volume 32 · Number 1 · December 2011 Inside this issue 1 Guest editorial: Prediabetes, CANRISK and screening in Canada 2 Nova Scotia Prediabetes Project: upstream screening and community intervention for prediabetes and undiagnosed type 2 diabetes 12 Piloting the CANRISK tool in Vancouver Coastal Health 19 Validating the CANRISK prognostic model for assessing diabetes risk in Canada’s multi-ethnic population 32 2008 Niday Perinatal Database quality audit: report of a quality assurance project 43 Identifying potentially eligible subjects for research: paper-based logs versus the hospital administrative database 47 Research methods of the Youth Smoking Survey (YSS) 55 Self-Monitoring Blood Glucose Workshop I: promoting meaningful dialogue and action at the provincial level 59 Self-Monitoring Blood Glucose Workshop II: development and dissemination of the DCPNS decision tool for self-monitoring blood glucose in non-insulin-using type 2 diabetes Chronic Diseases and Injuries in Canada a publication of the Public Health Agency of Canada Howard Morrison, PhD Editor-in-Chief (613) 941-1286 CDIC Editorial Board Lesley Doering, MSW Public Health Agency of Canada Robert Geneau, PhD Robert A. Spasoff, MD Associate Scientific Editor International Development Research Centre Isra Levy, MB, FRCPC, FACPM Claire Infante-Rivard, MD Associate Scientific Editor Ottawa Public Health Elizabeth Kristjansson, PhD Associate Scientific Editor Centers for Disease Control and Prevention Lesli Mitchell, MA Scott Patten, MD, PhD, FRCPC Michelle Tracy, MA Managing Editor University of Calgary Barry Pless, CM, MD, FRCPC Sylvain Desmarais, BA, BEd Assistant Managing Editor Montreal Children’s Hospital Kerry Robinson, PhD Public Health Agency of Canada Fabiola Tatone-Tokuda, MSc University of Ottawa Andreas T. Wielgosz, MD, PhD, FRCPC Public Health Agency of Canada Don Wigle, MD, PhD University of Ottawa Russell Wilkins, MUrb Chronic Diseases and Injuries in Canada (CDIC) is a quarterly scientific journal focussing on current evidence relevant to the control and prevention of chronic (i.e. noncommunicable) diseases and injuries in Canada. Since 1980 the journal has published a unique blend of peer-reviewed feature articles by authors from the public and private sectors and which may include research from such fields as epidemiology, public/community health, biostatistics, the behavioural sciences, and health services or economics. Only feature articles are peer reviewed. Authors retain responsibility for the content of their articles; the opinions expressed are not necessarily those of the CDIC editorial committee nor of the Public Health Agency of Canada. Chronic Diseases and Injuries in Canada Public Health Agency of Canada 785 Carling Avenue Address Locator 6807B Ottawa, Ontario K1A 0K9 Fax: (613) 941-2633 E-mail: [email protected] Indexed in Index Medicus/MEDLINE, SciSearch® and Journal Citation Reports/ Science Edition Statistics Canada To promote and protect the health of Canadians through leadership, partnership, innovation and action in public health. — Public Health Agency of Canada Published by authority of the Minister of Health. © Her Majesty the Queen in Right of Canada, represented by the Minister of Health, 2011 ISSN 1925-6515 This publication is also available online at www.publichealth.gc.ca/cdic Également disponible en français sous le titre : Maladies chroniques et blessures au Canada Guest editorial Prediabetes, CANRISK and screening in Canada David Butler-Jones, MD, Chief Public Health Officer, Public Health Agency of Canada Occurring as a result of both lifestyle and genetic factors, type 2 diabetes is a serious chronic disease that can give rise to complications including blindness, heart disease and kidney failure. About 2 million Canadians have been diagnosed with type 2 diabetes, but an estimated 400 000 who have the disease have not yet been diagnosed. A further estimated five million more have prediabetes, where blood sugar levels are elevated, but not high enough for a diabetes diagnosis. Diabetes often remains undetected for years before clinical diagnosis, and many newly diagnosed persons already exhibit signs of diabetic complications. The age-standardized prevalence of diagnosed diabetes has been climbing by an average of 7 percent per year over the past decade. While many lifestyle risk factors for diabetes are modifiable, for example, by increasing physical activity or losing excess weight, genetic factors such as family history and ethnicity cannot be changed. Yet even “non-modifiable” factors are important, since they interact with other risk factors to affect one’s overall diabetes risk. Risk assessment tries to weigh the combined effect of all possible risk factors, not only the obvious ones like obesity, gender and age. Risk assessment tools can help effectively and efficiently identify people at high risk who merit more conclusive diagnostic testing for diabetes and prediabetes. When coupled with proven lifestyle interventions, identifying those with prediabetes may help delay or even prevent disease progression to type 2 diabetes, while the early identification of those with diabetes may postpone or even avoid serious diabetes complications through timely clinical care. In this issue of Chronic Diseases and Injuries in Canada, three papers examine the theme of identifying people at high risk of diabetes and prediabetes using a new risk tool, CANRISK. Talbot and Dunbar invited participants in two rural Nova Scotia communities to self-administer the CANRISK questionnaire and take an oral glucose tolerance test, and then, if prediabetic, to take part in a Prediabetes Lifestyle Program. In Vancouver, Papineau 1 and Fong involved participants from East Asian, South Asian, Latin American and sub-Saharan African ethnic groups, as well as Caucasian and urban Aboriginal people. Robinson and colleagues provide evidence that CANRISK is a valid tool for assessing diabetes risk on a national scale for Canada’s multi-ethnic population. The papers in this issue clearly demonstrate that targeting those at risk of diabetes and prediabetes is both an essential and collaborative effort. These new developments, however, aren’t going to solve all our challenges. Encouraging the effective uptake of new tools like CANRISK is not the exclusive responsibility of the health care system or governments in general, nor is it the responsibility of those target groups at greatest risk. Rather, targeted prevention strategies are society-wide opportunities that call for all of us to share, promote and enable healthier lifestyles and enhanced prevention efforts. Let’s ensure it’s a collective effort. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Nova Scotia Prediabetes Project: upstream screening and community intervention for prediabetes and undiagnosed type 2 diabetes P. Talbot, MSc; M. J. Dunbar, MEd This article has been peer reviewed. Abstract Introduction: Identifying individuals in the prediabetic state may help delay/prevent disease progression to type 2 diabetes mellitus. We explored the feasibility of a household mailing approach for population-based screening of prediabetes and unidentified type 2 diabetes mellitus, developed standard protocol, and developed and implemented community-based lifestyle programs. Methods: The 16-item Canadian Diabetes Risk Assessment Questionnaire (CANRISK) was mailed to every household in two rural Nova Scotia communities. In total 417 participants aged 40 to 74 years with no prior diagnosis of diabetes self-administered the CANRISK and completed a 2-hour oral glucose tolerance test (OGTT) at a local health care facility. Those with prediabetes were invited to participate in a Prediabetes Lifestyle Program. Results: Glycemic status was identified as normal, prediabetes or diabetes for 84%, 13% and 3% of participants, respectively. Association between glycemic status and overall CANRISK risk score was statistically significant. Six CANRISK items were significantly associated with glycemic status: body mass index, waist circumference, history of hypertension and hyperglycemia, education and perceived health status. Participants and physicians gave positive feedback on the CANRISK screening process. Conclusion: The CANRISK holds promise as a population-based screening tool. Keywords: prediabetic state, hyperglycemia, primary prevention, health education, health behaviour, type 2 diabetes mellitus, lifestyle risk reduction, blood glucose Introduction According to the National Diabetes Surveillance System (NDSS), Nova Scotia (NS) has the second highest rate of diabetes mellitus (DM: type 1 and type 2 combined) in Canada.1 The crude prevalence of DM among NS adults aged over 19 years increased from 7.3% in 2001/2002 to 8.7% in 2005/2006.2 On average, 5000 individuals are referred to the province’s 39 Diabetes Centres (DCs) annually. The percentage of newly diagnosed cases presenting at DCs with prediabetes (PreDM) increased from 11.4% in 2003/2004 to 22.2% in 2007/2008. The 2003 and 2008 Canadian Diabetes Association Clinical Practice Guidelines support the need for early identification of PreDM and reinforce lifestyle and pharmacotherapy, but little has been stated regarding targets and recommended approaches.3,4 Consequently, the standard of care varies. Labelling individuals as having PreDM without offering appropriate care and guidance is also a concern. The mandate of the Diabetes Care Program of Nova Scotia (DCPNS), “to improve, through leadership and partnerships, the health of Nova Scotians living with, affected by, or at risk of developing diabetes,” includes standardizing the approach to DM care and education in NS by ensuring that DCs promote self-care, monitor the development and progression of DM complications, and follow national and provincial guidelines for optimal care. The DCPNS facilitates innovative, multi-site research by acting as a central co-ordination site, providing access to expert consultants in DM and DM surveillance; research design and ethics; and data collection, management, analysis and interpretation. In 2008, DCPNS released Prediabetes Guidelines for Nova Scotia to help standardize the approach to PreDM identification and intervention.5 These guidelines stress the importance of community-based programming aimed at preventing or delaying the onset of DM through modest weight reduction, healthful eating, physical activity, stress reduction and management, and the modification of cardiovascular risk factors. The Public Health Agency of Canada (PHAC) adapted the Canadian Diabetes Risk Assessment Questionnaire (CANRISK) from the Finnish Diabetes Risk Score (FINDRISC) questionnaire6 to identify individuals at high risk for developing DM.7 The DCPNS partnered with two District Health Authorities (DHAs) in rural NS to help validate the CANRISK for the Canadian population and to foster the development and implementation of two communitybased programs promoting lifestyle Author references: Diabetes Care Program of Nova Scotia, Halifax, Nova Scotia, Canada Correspondence: Pam Talbot, Suite 548 Bethune Building, 1276 South Park Street, Halifax, NS B3H 2Y9; Tel.: (902) 473-2622; Fax: (902) 473-3911; Email: [email protected] Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 2 changes known to prevent or delay the onset of type 2 DM among those with PreDM.8-11 Objectives Our project had two sets of objectives. In partnership with DHAs 1. explore the feasibility of a household mailing approach for populationbased screening of adults aged 40 to 74 years living in rural NS with the CANRISK by • evaluating the association between CANRISK responses and glycemic status, • examining the suitability of CANRISK items, and • exploring perceptions of participants and physicians about populationbased DM screening using the CANRISK and an oral glucose tolerance test (OGTT); 2. develop standard OGTT protocol for the project; and 3. develop and implement communitybased lifestyle programs for individuals identified as having PreDM. In partnership with PHAC, our objective was to pool NS data with data from other provinces to validate the CANRISK for the Canadian population. Methods Key local and provincial stakeholders were engaged early to reflect the realities of each community in the project design. Local advisory committees provided critical local context pertaining to the design and delivery of the PreDM screening and community-based lifestyle programs; a provincial advisory committee provided overall guidance for the project, facilitated joint decision-making between the project sites and helped build capacity to conduct applied research. The DCPNS Advisory Council provided advice regarding the implications of the project. This project conducted population-level screening for PreDM and undiagnosed DM using a mailed DM risk survey—the CANRISK—followed by an OGTT. Adults aged 40 to 74 years with no prior diagnosis of DM from Annapolis Valley Health (AVH) and Guysborough Antigonish Strait Health Authority (GASHA) self-administered the CANRISK and completed a 2-hour OGTT at a hospital laboratory or health centre. Feedback about the CANRISK screening process was collected from participants and physicians through self-administered surveys. Participants found to have PreDM were invited to take part in a community-based Prediabetes Lifestyle Program. The study protocol was approved by local DHA ethics committees, and all participants provided informed written consent. Recruitment Adults aged 40 to 74 years residing in the towns of Kentville / New Minas (in AVH) and Antigonish County (in GASHA) were targeted for participation. Individuals who already had DM or PreDM were excluded as were pregnant women who receive screening for gestational diabetes (GDM) as part of routine prenatal care. To raise awareness about the project prior to data collection, the project managers spoke about it at community events, physicians who championed the project discussed it with their colleagues and on the radio, and broadcast and print media ran advertisements about it. During initial recruitment (AVH: 2008-06-02 to 2008-07-08; GASHA: 2008-05-26 to 2008-08-28), study packages containing a one-page invitation, seven-page letter of information and consent, 16-item CANRISK and a measuring tape were distributed to every household in the town of Kentville (N = 3700) and the county of Antigonish (N = 6500) through the regular postal service as a bulk delivery (N = 10 200). Delivery was staggered so that the hospital labo ratories or health centres would not be overwhelmed by a high volume of participants scheduling tests. To increase enrolment, a second recruitment phase occured in AVH (2008-10-02 to 2008-11-05). A one-page flyer inviting residents to participate in the project and a one-page information sheet about PreDM were delivered to all households in the towns of Kentville and New Minas (N = 7391). Interested residents called the project manager to have a complete study package mailed to them. In GASHA (200809-29), 100 complete study packages were hand-delivered to residents of the Paq’tnkek First Nations Community. In total, 17 691 study packages were distributed (10 300 complete study packages and 7391 invitation flyers) at a cost of $7,560. CANRISK (NS version) Participants self-administered the CANRISK*. They could call the project manager of the Prediabetes Project for help if required. The CANRISK booklet did not include corresponding scores for the eight items derived from the FINDRISC; this scoring system6 was applied during data entry. Instructions on how to prepare for an OGTT were printed in the CANRISK booklet. Scores ranged from 0 to 26; a higher score represented a higher 10-year risk of developing type 2 DM (Table 1). The eight items added for CANRISK were not scored, but their association with the glycemic results was examined. The 16 CANRISK items included age group (0–4), body mass index (BMI: 0–3), waist circumference (0–4), physical activity (0–2), nutrition (0–1), history of hypertension (0–2) or hyper glycemia (0–5), family history of DM (0–5), mother’s ethnicity, father’s ethnicity, year of birth, education, perceived health, sex and, for women, history of GDM or large birth-weight babies. Laboratory procedure Potential participants gave verbal consent to participate in the study and then were booked for an OGTT. The project manager reviewed the OGTT preparation instructions with participants at this time and again when making a reminder call three days before their scheduled OGTT appointment. Participants were instructed to eat as usual for the three days prior to the OGTT * The CANRISK questionnaire used for this study is available in Appendix A (online only) from: http://www.phac-aspc.gc.ca/publicat/cdic-mcbc/32-1/ar-02-eng.php#ar0208. 3 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Table 1 Description of the scoring systema applied to the canrisk during data entry Score Risk category Proportion of people who will develop DM within 10 years 0–6 Low 1/100 7–11 Slight 1/25 12–14 Moderate 1/6 15–20 High 1/3 21–26 Very high 1/2 Abbreviations: CANRISK, Canadian Diabetes Risk Assessment Questionnaire; DM, diabetes mellitus; FINDRISC, Finnish Diabetes Risk Score. a Adapted from FINDRISC.6 and then fast (no food or drink, except for sips of water) for at least 8 hours before the test. Upon arriving at the participating hospital or health centre, participants signed an informed consent form. They then had a 4 ml venous blood sample drawn for a fasting plasma glucose (FPG) test. A phlebotomist or certified lab technician tested their capillary blood glucose (CBG) by collecting a single drop of blood using a lancet and tested this with a CBG meter. Participants with a CBG less than 7.0 mmol/L completed a 75 g OGTT. Participants remained on-site, sedentary, and neither eating nor smoking for two hours. They then had a 4 ml venous blood sample drawn for their 2-hour plasma glucose (2hPG), after which they were offered fruit juice and a snack. Participants with a fasting CBG equal or greater than 7.0 mmol/L did not complete the OGTT but were referred to their family physician (FP) for appropriate follow-up care. These participants were not excluded from the study. All specimens were centrifuged and analyzed as per the test tube manufacturer’s guidelines. Statistical analyses Descriptive statistics were computed to describe the participants by site. A Pearson chi-square (χ2) test was computed to assess the association between CANRISK risk category and glycemic status, and a series of Pearson chi-square tests were computed to assess the association between each CANRISK item and glycemic status. All analyses were conducted using Statistical Package for Social Sciences (SPSS) version 15.0 for Windows (SPSS, Chicago, Il). Participant feedback Results The project managers provided participants with their blood test results in writing or verbally as well as appropriate recom mendations based on the results. They also mailed them an anonymous self-administered Participant Feedback Form. This addressed participants’ awareness of the project, prior know ledge of PreDM, ability to understand the CANRISK and OGTT preparation instructions, concerns about having PreDM or DM before and after participation in the project, and reasons for participating in the study. Study sample In total, 417 adults aged 40 to 74 years living in AVH (n = 186; 45%) or GASHA (n = 231; 55%) participated in the NS Prediabetes Project (initial recruitment: n = 335; second recruitment: n = 82). Approximately 70% of participants (n = 289) were women, over 95% (n = 397) reported having only White ancestry, and nearly 40% held a post-secondary diploma (n = 10; 2%) or degree (n = 156; 37%). Of the 411 parti cipants who reported year of birth, the average age was approximately 57 years (men: 58 years; women: 56 years). Physician feedback After the data collection, physicians from each project site (Kentville/New Minas: n = 40; Antigonish County: n = 74) were invited to contribute their thoughts about the project by responding anonymously to a three-item Physician Feedback Form. This form asked them how the PreDM screening had impacted their work, whether the CANRISK should be used to screen for PreDM or DM and about their awareness of community-based programs promoting healthy lifestyle choices. Of the 417 participants, 416 completed all (n = 400; 96%) or part (n = 16; 4%) of the CANRISK, all completed an FPG test and CBG reading and 399 (96%) completed an OGTT. Approximately 5% of participants (n = 22) had a CBG equal or greater than 7.0 mmol/L at their initial OGTT appointment and were ineligible to receive the 75 g Trutol drink at that visit; four of these participants completed the protocol on a different day. One participant was unable to retain the Trutol drink at the initial appointment but completed the protocol on a different day. Prediabetes Lifestyle Program Case ascertainment The project managers worked with existing resources and personnel within their communities to develop and deliver a PreDM Lifestyle Program (Appendix C). All participants identified as having PreDM were invited to take part in the Program. Approximately 84% (n = 350) of parti cipants had normal blood glucose levels, 13% (n = 54) had blood glucose in the PreDM range and 3% (n = 13) had blood glucose in the DM range. Within the PreDM group, the percentage of cases with isolated impaired fasting glucose Glycemic status Glycemic status (i.e., normal, PreDM or DM) was determined using the most complete data possible. When available, FPG and 2hPG readings were combined to derive glycemic status; otherwise FPG was used alone (Appendix B). Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 4 (IFG), isolated impaired glucose tolerance (IGT) or IFG/IGT combined was 48%, 41% and 11% respectively. CANRISK profile A CANRISK score was calculated for the 400 participants who completed all items on the CANRISK; scores ranged between 0 and 25. There was a significant association between participants’ glycemic status and their CANRISK risk category (p < .01). Approximately 98% of participants in the low-risk category, compared to 46% in the very high-risk category, had blood glucose in the normal range (Figure 1). Approximately 23% of individuals with blood glucose in the normal range had a high to very high CANRISK score, compared to 64% of those in the PreDM range and 58% of those in the DM range (Figure 2). There was a significant association between participants’ glycemic status and six of the CANRISK items: BMI, waist circumference, history of hypertension, history of hyperglycemia, post-secondary education and perceived health status (Table 2). Although not statistically significant, there were trends in the expected direction for six additional items: daily physical activity, daily fruit and vegetable consumption, family history of DM, history of GDM and history of high birth-weight babies (> 4 kg) among women, and sex (19% versus 15% with blood glucose in PreDM or DM range for males and females, respectively). There was no significant association or trend for age group and ethnicity (Table 2). Participant feedback Approximately 62% of participants (n = 257) returned a Participant Feedback Form (AVH: 75%; GASHA: 51%). The following results pertain only to those who completed this form. We cannot compare the characteristics of these respondents to those of non-respondents as the Feedback Form was anonymous. Approximately 42% of Participant Feedback Form respondents (n = 109) indicated that they had heard about the project before receiving the study package. The most commonly cited sources of this information were the newspaper (28%), work (24%), friends and family (23%) and the radio (22%); less common sources included notices in doctor’s offices (6%) and community boards, grocery store flyers, church bulletins, and community television ads (all ≤5%). Nearly all respondents (n = 252; 98%) reported being able to complete the CANRISK on their own. All respondents agreed that the OGTT instructions were not difficult to understand, with 85% rating them as very easy to understand. Approximately 53% of respondents (n = 136) indicated that they knew what PreDM was prior to receiving the study package, and approximately 62% (n = 160) indicated that they were not worried about having PreDM or DM at any time. Of the 96 respondents who reported that they worried about having PreDM or DM at some point, 73% (n = 70) were worried before the study Nearly all respondents (n = 252; 98%) indicated why they took part in the study: 48% (n = 124) wanted to be tested, 41% (n = 106) wanted to help the study and 41% (n = 106) had a family history of DM. Physician feedback Approximately 22% of physicians (n = 25) returned a Physician Feedback Form (AVH: 33%; GASHA: 16%). Of the 25 responding physicians, 40% (n = 10) indicated that the CANRISK screening process had no impact on their work, and 60% (n = 15) indicated that there was a minimal to moderate impact. When asked how the CANRISK screening process affected their work, these 15 physicians described two main effects: that it provided an opportunity to speak about positive lifestyle changes with patients (n = 7; 47%) and that it identified previously undiagnosed cases of PreDM or DM (n = 6; 40%). Other less common examples included more office visits, that patients asked more informed questions about PreDM or DM, that it encouraged patients to take charge of their health behaviours and that there were more phone calls (all ≤ 33%). When asked if the CANRISK should be used to screen for DM in their community, 52% (n = 13) replied “yes,” 28% (n = 7) replied “no” and the remainder were undecided or did not respond. Figure 2 Percentage of participants in each CANRISK risk category by glycemic status 100% 100% 80% 80% Percentage Percentage Figure 1 Percentage of participants in each glycemic status category by CANRISK risk category package arrived and 27% (n = 26) were not worried before the study package arrived but started to worry after completing the CANRISK and OGTT. 60% 40% 60% 40% 20% 20% 0% 0% Low (0–6) Slight (7–11) Moderate (12–14) High (15–20) Normal Very High (20–26) CANRISK risk category Normal Prediabetes Prediabetes Diabetes Glycemic Status Low (0–6) Diabetes Slight (7–11) High (15–20) 5 Moderate (12–14) Very High (21–26) Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Table 2 Percent frequency of response options for CANRISK items by glycemic status χ2 Glycemic status, Normal PreDM DM p-value BMI (kg/m ) ≥ 25 (n = 281/415) 64.2 86.8 84.6 < .01 Waist circumference (> 35 in / 88 cm for women; > 40 in / 102 cm for men) (n = 225/410) 50.9 80.8 58.3 < .01 History of hypertension (n = 135/414) 28.7 52.8 53.8 < .01 History of hyperglycemia (n = 39/410) 7.2 17.6 38.5 < .01 Post-secondary degree/diploma (n = 166/415) 42.7 30.2 7.7 .04 Excellent / very good perceived health (n = 227/414) 59.2 34.0 23.1 < .01 Engaged in daily physical activity (n = 248/412) 62.4 49.1 46.2 .10 Ate fruits and vegetables daily (n = 350/414) 85.6 81.1 69.2 .21 ≥ 1 first degree relative with DMc (n = 229/416) 52.3 69.8 69.2 .08 6.1 11.8 12.5 .40 History of large birth-weight (> 9 pounds / 4 kg) baby (n = 50/288 females) 16.7 20.6 25.0 .72 Age 45–64 years (n = 292/416) 70.9 62.3 84.7 .45 White ethnicity for mother and father (n = 397/411) 96.5 96.1 100.0 .78 Female (n = 289/416) 70.6 64.2 61.5 .52 CANRISK response option a 2 b History of GDM (n = 20/287 females) Abbreviations: BMI, body mass index; CANRISK, Canadian Diabetes Risk Assessment Questionnaire; DM, diabetes mellitus; GDM, gestational diabetes; PreDM, prediabetes. Note: This table contains responses to 14 CANRISK items of the 16 in the questionnaire. White ethnicity combines two CANRISK items: mother’s and father’s ethnicities. Year of birth is a continuous variable and was therefore not analyzed. Number of participants who selected an option as a proportion of the number who completed the item in the CANRISK survey. a n = 10 participants added post-secondary diploma as an option; all 10 had normal blood glucose levels. b Based on “yes” response to family history of DM: parent, sibling or child having DM, non-response (11%, 8%, 17%, respectively) assumed to be “no.” c Most responding physicians (n = 21; 84%) indicated that they were aware of programs in the community that promoted healthy lifestyle choices and indicated that they recommended these programs to their patients with PreDM or DM. Prediabetes Lifestyle Program Each project site developed a Prediabetes Lifestyle Program that included five core components addressing lifestyle factors known to prevent or delay8-11 the development of type 2 DM among at-risk individuals (Appendix C). The 54 individuals identified as having PreDM were invited to take part in a community-based PreDM Lifestyle Program; 19 (35%) did so. Discussion Population-level screening process This project provided an opportunity to conduct population-level screening for PreDM and undiagnosed DM using a mailed self-administered DM risk survey. A mail-out approach rather than one-on-one recruitment was used as it more closely mirrored the context within which the CANRISK would be used if adopted by the province, especially given the current climate of fiscal restraint and limited health care human resources. The project team was cognisant of the need to contain expenses and not infringe on the workloads of already overburdened FPs. Over 10 000 CANRISK questionnaires were distributed in the pilot communities, and 417 residents were screened in seven months by two part-time (0.5 full-time equivalent) project managers using the existing laboratory infrastructure. The distribution cost was approximately $0.43 per package, and the overall cost was $18.13 per screened participant. the screening pilot. It is possible that the two-hour time commitment coupled with a seven-page letter of information and consent may have overwhelmed potential participants, thus negatively impacting the participation rate. Based on 2006 Census estimates, approxi mately 14 600 residents in the pilot communities were between 40 and 74 years of age.12-14 Approximately 3% of this eligible population participated in If adopted as part of a chronic disease prevention strategy, the CANRISK would be only one facet of a multi-faceted approach. It is not reasonable to assume the study response rate would be Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 6 In survey research, a low response rate typically limits the generalizability of findings. Study participants were more educated than the general population, possibly resulting in lower case ascer tainment. Although the distribution of CANRISK scores in the study sample may not be representative of that in the general population, there is no reason to believe that the actual CANRISK responses would correlate differently with blood glucose values for study participants than for the general population. replicated if the CANRISK were to be used as part of a province-wide initiative. Ideally, the CANRISK would be widely available through multiple venues (e.g. Internet, newspaper, insert with healthcard renewal form, physician offices, etc.) with the hope that people would fill it out and that those who score high would speak to their FP about having their blood glucose tested. To reach more vulnerable and underserved populations, alternative strategies would need to be used. other regions in the province. Also, the DC at each site offers PreDM programming aimed at delaying or preventing the development of DM. Finally, participants were highly educated with 37% holding a post-secondary degree, compared to 22% of the general NS population.16 Education is a well-known determinant of health with increasing levels of education equating to better health. CANRISK Case ascertainment Overall, 84% of participants had blood glucose levels in the normal range, 13% in the PreDM range and 3% in the DM range. These sites in Nova Scotia had a slightly higher percentage of participants with normoglycemia compared to the percentage for the first wave sites combined15 in New Brunswick, Prince Edward Island and Saskatchewan (79%). The distribution of participants within the PreDM group also differed for NS compared to the first wave sites. In NS, the percentages of IFG and IGT cases within the PreDM group were similar at 48% and 41%, respectively, compared to 29% and 59% for the first wave sites.15 The percentage of IFG/IGT cases within the PreDM group was similar for NS and the first wave sites at 11% and 12%, respectively.15 There are several possible explanations for the observed differences. Despite variable practice across the province, the NS project sites used uniform OGTT protocol, requiring standard preparation for the three days preceding the OGTT. These protocols were printed in the CANRISK booklet and orally communicated to participants at the time of their OGTT booking and during a reminder call three days before their OGTT. During the OGTT, participants were required to remain sedentary and non-smoking on-site for two hours between administering the 75 g Trutol and the 2hPG collection. The project sites were considered to be well staffed with physicians, and all participants had an FP at enrolment. Both sites have a regional hospital, increasing participants’ access to FPs and specialists, compared to The NS project team did not include the FINDRISC scoring system6 on the self-administered CANRISK for several reasons: • Although slightly different versions of the FINDRISC have been validated for European and Mediterranean populations,17-19 differences in the ethnic composition, lifestyle, and genetic and environmental exposures in Canada warranted that FINDRISC cut-off points and relative weights be validated for the Canadian population before being put into use.19-24 • Misclassification based on the Finnish scores could have caused participants to worry needlessly. • Not all CANRISK items had a corresponding score, possibly leading to participant confusion or response bias. • The interpretation of the 10-year DM risk requires a high degree of literacy or numeracy. During analysis, a CANRISK score was calculated based on the Finnish scoring system,6 and it was significantly associated with glycemic status. Based on this observation, the Finnish scoring system6 could be used for the CANRISK until a Canadian scoring system is devised, but some effort should be made to determine how well individuals understand the risk scores. When examined individually, six CANRISK items were significantly associated with participants’ current glycemic status; six additional items showed a trend in the expected direction. For these six, the lack of significance might be the result of low power due to the small sample size rather than a true lack of association. 7 Modifications to the CANRISK format could improve the completeness and accuracy of data collected. Approximately 11% of participants (n = 46) recorded their waist circumference range but not their waist circumference measurement. The waist circumference measurement could be omitted from future versions of the CANRISK as risk is assigned based on the range. Most participants (98%) reported a waist circumference range; however, the accuracy of this measure may be suspect. A high percentage (> 80%) of those with blood glucose in the PreDM range reported having a waist circumference more than 35 inches (88 cm) for females or more than 40 inches (101 cm) for males; however, for those in the DM range, this percentage was much lower (58%). This unexpected finding may be a result of the small number of participants in the DM group (n = 13). However, this pattern was not observed for BMI, an alternative measure of obesity. A similar percentage (> 84%) of participants in the PreDM and DM groups had a BMI over 25 kg/m2. The waist circumference item will need to be examined in more detail using the pooled national dataset. The greatest non-response rate for a CANRISK item was for the one addressing family history of DM. The item requires that participants check “yes,” “no” or “don’t know” for five different familial relationships: mother, father, siblings, children and other; however, the only response that adds to the risk score is “yes.” This item could be simplified by requesting participants to check all the family members that have DM. Approximately 3.5% of female participants (n = 10) did not respond to the items addressing GDM and/or giving birth to a large baby, 8 of these women indicated that the items were not applicable. Forcing women to choose between yes or no for these items implies that all female respondents must have been pregnant or given birth at some time. A third option of “not applicable” would alleviate this problem and make the items more sensitive toward women who have neither been pregnant nor given birth. The “not applicable” option would also apply to Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada women who have not been screened for GDM, especially those in older age groups who would have been screened at their FP’s discretion. Participant feedback Although not all participants completed the Participant Feedback Form, the 62% (n = 257) who did indicated that the CANRISK screening process was generally positive. Participants found the CANRISK and OGTT protocol easy to understand, a fact that likely reflects the high educational attainment of participants as well as local enhancements to formatting that improved the CANRISK’s appearance and readability. Approximately half the participants who responded to the Participant Feedback Form indicated that they knew what PreDM was prior to receiving a study package. Recognising that the risk for adverse health outcomes may be higher among those who do not access health care services on a regular basis, NS opted to use a mail-out approach to participant recruitment. In this way, a broad population was reached with educational literature about PreDM and its risk factors. Every household in the two project sites received a study package, regardless of the residents’ eligibility to take part in the study. Physician feedback In the planning stages of the project, FPs expressed concern about the impact of the study on their workload. These concerns proved to be unfounded. Approximately 92% of the 25 physicians who responded to the Physician Feedback Form indicated that the CANRISK screening had little to no impact on their workload. When specific impacts were noted, many were positive; for example, the study provided an opportunity to discuss positive lifestyle choices with patients, or the screening identified previously undiagnosed cases of PreDM and DM. Although the responses received were overwhelmingly positive, it should be noted that the response rate for the Physician Feedback Form was fairly low (22%). Prediabetes Lifestyle Program It was hoped that a “real world” program that reflected community realities and partners would be developed by mobilizing available community resources, become part of the standard of care within the community, and serve as a template for the development of similar programs across the province. However, the 12-month funding window did not allow sufficient time to build the partnerships necessary to develop and sustain this type of programming. Although the initial vision of the Prediabetes Lifestyle Program was not fully realized in this project, important groundwork was established. The successful partnership with DHAs resulted in a willingness to continue the work started through this project. With funding from PHAC-Atlantic Region (2009/2010 and 2010/2011) and in partnership with local and provincial stakeholders, AVH Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 8 developed and evaluated a comprehensive and sustainable community-based lifestyle program for people with PreDM, other at-risk populations and individuals in the early stages of chronic disease. Acknowledgements The Diabetes Care Program of Nova Scotia would like to acknowledge and thank those directly involved in the development and implementation of the NS Prediabetes Project. The successful completion of this project would not have been possible without the tireless efforts of the project managers and volunteer committee members both at the local and provincial level. We would like to thank Jennifer Payne for her helpful editorial comments and critical feedback regarding this manuscript. We also want to acknowledge the important insights from the First Wave project teams in New Brunswick, Prince Edward Island and Saskatchewan. Finally, we would like to thank the Public Health Agency of Canada and the Prediabetes Technical Advisory Group for the funding, central support, and guidance provided throughout the NS Prediabetes Project process. Appendices Appendix B Definitions for glycemic status Table B1 Glycemic status based on fasting plasma glucose and 2-hour plasma glucose Classification FPG, mmol/L Normoglycemia Isolated IFG Prediabetes Diabetes Table B2 Glycemic status based on fasting plasma glucose test 2hPG, mmol/L Classification < 6.1 and < 7.8 Normoglycemia 6.1–6.9 and < 7.8 Prediabetes Isolated IGT < 6.1 and 7.8–11.0 IFG & IGT 6.1–6.9 and 7.8–11.0 ≥ 7.0 or ≥ 11.1 FPG, mmol/L < 6.1 6.1–6.9 Diabetes ≥ 7.0 Abbreviations: FPG, fasting plasma glucose. Abbreviations: FPG, fasting plasma glucose; 2hPG, 2-hour plasma glucose; IFG, impaired fasting glucose; IGT, impaired glucose tolerance. Appendix C Nova Scotia Prediabetes Project – Prediabetes Lifestyle Projects A major objective of the NS Prediabetes Project was to explore, develop and implement a community-based lifestyle program for at-risk individuals, including those with PreDM. The project managers worked with community partners and health care personnel to identify and mobilize available community resources. The Prediabetes Lifestyle Programs developed as part of this project included five core components, which were presented at both screening sites, Annapolis Valley Health (AVH) and Guysborough Antigonish Strait Health Authority (GASHA): 1. Prediabetes education: This component focused on the importance of making healthy lifestyle choices to prevent or delay the onset of DM. It explained the risk factors for developing DM, criteria used to diagnose DM, prevention and treatment of DM and healthful eating. • AVH: Presented by a certified diabetes educator (CDE) at Valley Regional Hospital (VRH). • GASHA: Presented by a CDE at Health Connections, a community space designated for health-related education and programs. 2. Goal setting: This component focused on factors that help people effect change, challenges to meeting goals and setting specific, measurable, attainable, relevant and time-bound (SMART) goals. Participants could set an achievable and meaningful goal. • AVH: Presented by a professional psychologist at VRH. • GASHAa: Presented by a health motivator at Health Connections. 3. Nutrition: This component focused on information about how to read labels and choose healthier foods and discussed topics such as sodium, fats, and fibre. • AVH: Presented by a community dietitian at VRHb. • GASHA: Presented by a public health dietitian at Health Connections. 4. Physical activity: This component focused on exercise suitable for those who may have been inactive for some time. Participants learned about the value of walking and were instructed how to use a pedometer. • AVH: Presented by a professional kinesiologist / trained exercise instructor at VRH (Cardiac Rehab). • GASHA: Presented by the Director of the Antigonish Recreation Department at Health Connections. 5. Stress management: This component focused on stress symptoms, stressors, and stress management. • AVH: Presented by a professional psychologist at VRH. • GASHAa: Presented by a health motivator at Health Connections. a Goal setting and stress management were delivered as a combined session in GASHA. b This session was to be delivered by a dietitian from one of the local grocery stores; however, by the time the session was delivered, the grocery chain had laid off all their staff dietitians in many rural locations. 9 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada References 1. Diabetes in Canada: highlights from the National Diabetes Surveillance System, 2004-2005. Ottawa (ON): Public Health Agency of Canada; 2008. [Catalogue No.: HP32-2/2005 | 978-0-662-05378-1]. 2. Diabetes Care Program of Nova Scotia. Nova Scotia diabetes statistics report 2008 [Internet]. Halifax (NS): Diabetes Care Program of Nova Scotia; 2008 Mar [cited 2010 May 05]. Available from: http:// www.diabetescareprogram.ns.ca/statistics /Diabetes_Stats_Report_2008.pdf 3. Canadian Diabetes Association Clinical Practice Guidelines Expert Committee. Canadian Diabetes Association 2003 clinical practice guidelines for the prevention and management of diabetes in Canada. Can J Diabetes. 2003;27(suppl 2):1-152. 4. Canadian Diabetes Association Clinical Practice Guidelines Expert Committee. Canadian Diabetes Association 2008 clinical practice guidelines for the prevention and management of diabetes in Canada. Can J Diabetes. 2008;32(suppl 1):S1-S201. 5. Diabetes Care Program of Nova Scotia. Prediabetes (PreDM) screening and management of high-risk individuals (adults) [Internet]. Halifax (NS): Diabetes Care Program of Nova Scotia; 2008 [cited 2010 May 05]. Available from: http://www.diabetescareprogram.ns.ca /guidelines.asp 6. Tuomilehto J, Lindström J. Type 2 diabetes risk assessment form [Internet]. Finland: Finnish Diabetes Association; 2001 [cited 2011 Jul 05]. Available from: h t t p : / / w w w. d i a b e t e s . f i / f i l e s / 110 0 /Type2diabetesRiskTest_.jpg 7. Kaczorowski J, Robinson C, Nerenberg K. Development of the CANRISK questionnaire to screen for prediabetes and undiagnosed type 2 diabetes. Can J Diabetes. 2009; 33(4):381-5. 8. Tuomilehto J, Lindström J, Errikson JG, Valle TT, Hämäläinen H, Ilanne-Parikka P, et al. Prevention of type 2 diabetes mellitus by changes in lifestyle among subjects with impaired glucose tolerance. N Engl J Med. 2001;344(18):1343-50. 9. Knowler WC, Barrett-Connor E, Fowler SE, Hamman RF, Lachin JM, Walker EA, et al. The Diabetes Prevention Program Research Group. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med. 2002;346(6):393-403. 10. Ramachandran A, Snehalatha C, Mary S, Mukesh B, Bhaskar AD, Vijay V. The Indian Diabetes Prevention Programme shows that lifestyle modification and metformin prevent type 2 diabetes in Asian Indian subjects with impaired glucose tolerance (IDPP-1). Diabetologia. 2006;49(2):289-97. 11. Pan XR, Li GW, Hu YH, Wang JX, Yang WY, An ZX, et al. Effects of diet and exercise in preventing NIDDM in people with impaired glucose tolerance. The Da Qing IGT and Diabetes study. Diabetes Care. 1997;20(4):537-44. 12. Nova Scotia Community Counts [Internet]. Province of Nova Scotia 2010. Demographics. Population by 5-Year Age Groups: Antigonish County [cited 2010 Sep 27]; [one screen]. Available from: http://www.gov.ns.ca /finance/communitycounts/dataview .asp?gnew=&table=tabled2&acctype=0& chartid=&mapid=&dcol=&sub=&ptype= geo&group=&group1=&group2=&group3=& group4=&range=&gview=3&glevel= cnt&gnum=cnt1214 13. Nova Scotia Community Counts [Internet]. Province of Nova Scotia 2010. Demographics. Population by 5-Year Age Groups: Kentville [cited 2010 Sep 27]; [one screen]. Available from: http://www.gov.ns.ca /finance/communitycounts /dataview.asp?gnew=&table=table_d2& acctype=0&chartid=&mapid=&dcol= &sub=&ptype=geo&group=&group1= &group2=&group3=&group4=&range= &gview=3&glevel=com&gnum=com715 14. Nova Scotia Community Counts [Internet]. Province of Nova Scotia 2010. Demographics. Population by 5-Year Age Groups: New Minas [cited 2010 Sep 27]; [one screen]. Available from: http://www.gov.ns.ca/finance /communitycounts/dataview.asp?gnew=& table=table_d2&acctype=0&chartid= &mapid=&dcol=&sub=&ptype=geo& group=&group1=&group2=&group3= &group4=&range=&gview=3&glevel= com&gnum=com720 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 10 15. McGibbon A, Tuttle J, Amirault D, McCardle LA, Leuschen L, Robinson C, et al. A pilot study using the CANRISK survey to identify prediabetes and diabetes in Canada. Proceedings of the 3rd International Congress on Prediabetes and the Metabolic Syndrome; 2009 Apr 3-5; Nice (FR). J Diabetes. 2009 April;1(s1):A43. 16. Nova Scotia Community Counts [Internet]. Province of Nova Scotia 2010. Education: educational attainment (20 years and over) Nova Scotia [cited 2010 Sep 27]; [one screen]. Available from: http://www.gov.ns.ca /finance/communitycounts/dataview.asp?gnum =pro9012&gnum2=pro9012&gname=& gview=3&glevel=pro>ype=&ptype= &gsel=&table=table_e1b&acctype=2 17. Franciosi M, De Berardis G, Rossi MC, Sacco M, Belfiglio M, Pellegrini F, et al. Use of the diabetes risk score for opportunistic screening of undiagnosed diabetes and impaired glucose tolerance: the IGLOO (Impaired Glucose Tolerance and LongTerm Outcomes Observational) study. Diabetes Care. 2005 May;28(5):1187-94. 18. Bergmann A, Li J, Wang L, Schulze J, Bornstein SR, Schwarz PE. A simplified Finnish diabetes risk score to predict type 2 diabetes risk and disease evolution in a German population. Horm Metab Res. 2007 Sep;39(9):677-82. 19. Li J, Bergmann A, Reimann M, Bornstein SR, Schwarz PE. A more simplified Finnish diabetes risk score for opportunistic screening of undiagnosed type 2 diabetes in a German population with a family history of the metabolic syndrome. Horm Metab Res. 2009 Feb;41(2):98-103. 20. Saaristo T, Peltonen M, Lindström J, Saarikoski L, Sundvall J, Eriksson JG, et al. Cross-sectional evaluation of the Finnish Diabetes Risk Score: a tool to identify undetected type 2 diabetes, abnormal glucose tolerance and metabolic syndrome. Diab Vasc Dis Res. 2005 May;2(2):67-72. 21. Silventoinen K, Pankow J, Lindström J, Jousilahti P, Hu G, Tuomilehto J. The validity of the Finnish Diabetes Risk Score for the prediction of the incidence of coronary heart disease and stroke, and total mortality. Eur J Cardiovasc Prev Rehabil. 2005 Oct;12(5):451-8. 22. Schwarz PE, Li J, Lindström J, Tuomilehto J. Tools for predicting the risk of type 2 diabetes in daily practice. Horm Metab Res. 2009 Feb;41(2):86-97. 23. Schwarz PE, Li J, Reimann M, Schutte AE, Bergmann A, Hanefeld M, et al. The Finnish Diabetes Risk Score is associated with insulin resistance and progression towards type 2 diabetes. J Clin Endocrinol Metab. 2009 Mar;94(3):920-6. 24. Gao WG, Dong YH, Pang ZC, Nan HR, Wang SJ, Ren J, et al. A simple Chinese risk score for undiagnosed diabetes. Diabet Med. 2010 Mar;27(3):274-81. 11 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Piloting the CANRISK tool in Vancouver Coastal Health D. Papineau, PhD; M. Fong, MSN This article has been peer reviewed. Abstract Introduction: Vancouver Coastal Health Authority’s Healthy Living Program implemented this pilot study to test and validate the Canadian Diabetes Risk Assessment Questionnaire (CANRISK) developed by the Public Health Agency of Canada as a screening tool for undiagnosed type 2 diabetes mellitus (DM) and prediabetes. Key objectives were to test the feasibility and acceptability of screening urban ethnic groups using the CANRISK, increase awareness of risk factors for DM and preDM and develop resources for lifestyle change. Methods: The study recruited participants through community groups and churches, intraorganizational emails, primary care clinics and word of mouth. They completed the CANRISK and an oral glucose tolerance test (OGTT) either individually or as part of a group. Groups received a brief diabetes prevention information session. Documents to support lifestyle change were distributed to all participants. Results: Participants (n = 556) were recruited among East Asian, Caucasian, South Asian and Latin American ethnic groups. Of these, 17% had OGTT results in the preDM range and 3% in the DM range. Over 90% of participants reported that the CANRISK wording was clear and that they had received useful information about lowering their diabetes risk. Conclusion: The benefit of using an OGTT was in identifying 11% of the sample of participants who had impaired glucose tolerance (IGT) and did not show abnormal fasting plasma glucose (FPG) results. All participants with abnormal laboratory results were provided with follow-up educational interventions in their own language. Keywords: diabetes, prediabetes, patient recruitment, oral glucose tolerance test, OGTT, ethnicity, prevention Introduction This provincial pilot study aims to test and validate the Canadian Diabetes Risk Assessment Questionnaire (CANRISK)* developed by the Public Health Agency of Canada (PHAC) as a screening approach for undiagnosed type 2 diabetes mellitus (DM) and prediabetes (preDM).1 The pilot was implemented by the Vancouver Coastal Health Authority’s (VCH) Healthy Living Program (HLP). The Program provides health promotion and chronic disease prevention services for adults who are well, at-risk for chronic diseases or recently diagnosed with a chronic disease. Their life circumstances include one or more of the following: low income; low level of education; immigrant; Aboriginal ancestry; and social isolation and/or marginalization. Strategies used to identify and support these individuals include screening, health promotion and self-management support. The objectives of the pilot study were to • test the feasibility and acceptability of screening urban ethnic groups using the CANRISK; • identify, develop and provide resources to support lifestyle changes; • enhance partnerships and collaborate with community organizations to increase awareness and screen for DM and preDM; • develop partnerships and linkages with family physicians; • evaluate satisfaction and acceptability of screening activities among the target groups and health care providers; and • increase research participants’ knowledge of risk factors for preDM and DM and provide resources for lifestyle change. Methods Participants Pilot study participants were aged 30 to 74 years, able to provide informed consent, and neither pregnant nor diagnosed with DM. At the request of PHAC, the pilot study targeted members of the following ethnic communities: East Asian (Chinese, Vietnamese, Filipino); South Asian (Punjabi); Latin American; and sub-Saharan African, though Caucasians and urban Aboriginals were also approached to participate. At the Vancouver site, we broadened the CANRISK survey’s age range (40 to 74 years) to include those aged 30 to 39 years as several of the targeted ethnic groups have a higher genetic risk of developing DM2,3 compared with Caucasians. This was also based on the Canadian Diabetes Association’s (CDA’s) recommendation * This version of the questionnaire is available online from: http://www.diabetes.ca/documents/for-professionals/NBI-CANRISK.pdf. Author references: Vancouver Coastal Health, Vancouver, British Columbia, Canada Correspondence: Maylene Fong, Vancouver Coastal Health, Healthy Living Program, 5913 West Boulevard, Vancouver, BC V6M 3X1; Tel.: (604) 267-4433; Fax: (604) 267-3993; Email: [email protected] Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 12 that those with one or more of the 13 risk factors be tested earlier than age 40 years.4 CANRISK group session and invite their members to participate in the pilot study or else invite the pilot study team to recruit on site during a local event. Recruitment Once granted ethics approval by the University of British Columbia (UBC) and the VCH Research Institute (VCHRI), enrolment ran from June 2009 to May 2010. The core research team involved in the recruitment and implementation were registered professional staff (nurses or dietitians). Documents informing potential participants about the study were developed and translated. Several strategies were used to recruit participants. Family physicians in private practice were identified as key partners. The study team gave presentations at physician education sessions and at VCH primary health care clinic team meetings. As a result, 49 physicians, 3 residents, 4 nurse practitioners and 3 nurses working in physician offices agreed to refer patients to the study. The study team met with key VCH leaders to discuss how best to inform VCH staff, many of whom had risk factors for diabetes, about the study. An email was sent to all VCH staff about the opportunity to participate in the study. Leaders from residential care and assisted living sites agreed to circulate study brochures and display recruitment posters in staff rooms. Leaders providing education/clinical services to adults, older adults and parents in nine locations also agreed to collaborate. A recruitment partnership was established with UBC researchers to target the Latin American community. Partnerships with community organizations and churches that support ethnic communities or low-income populations were the most successful at recruiting study participants. As part of their ongoing work, the study team had established relationships with several community groups to collaborate in diabetes prevention events. Staff in these organizations would plan a Team members also routinely asked participants to mention the study to friends and family. CANRISK administration Different options were offered to complete the study protocol while meeting the varied needs and preferences of participants. The protocol included the following steps: (1) fill out the CANRISK questionnaire; (2) complete an oral glucose tolerance test (OGTT) and a hemoglobin A1C (HbA1C) test; (3) receive test results, with all necessary explanations, over the telephone from a member of the pilot study team, followed by a mailed copy of the test results. Two screening events (16 and 23 participants) combined CANRISK completion, education on preventing DM and laboratory staff performing OGTT and HbA1C testing on site. These were held in Spanish and in Vietnamese. There were 36 group events where the CANRISK was completed with a brief introduction to diabetes prevention. Participants then went individually to the laboratory for an OGTT. These groups ranged in size from 5 to 25 participants and were held in Cantonese, Mandarin, Punjabi or English in various locations including churches, municipal community centres and community organization offices. Twelve volunteers offered support with groups. Another participant subset completed the CANRISK as part of an individual appointment with a team staff member and then went to the laboratory on a different day. Laboratory protocol for OGTT and HbA1C testing Study funding was used towards two commercial laboratories performing the blood tests. A partner physician from the VCH Primary Care Network agreed to block order the laboratory tests. Participants were provided with a standard set of instructions on how to prepare for the OGTT. The team reviewed the 13 laboratories’ testing and analysis protocol for conformity with the documentation provided by PHAC regarding OGTT and HbA1C. They were found to meet the requirements. Lifestyle intervention First, the pilot study team reviewed the documentation, health services and community supports available for future participants in making healthy lifestyle changes linked to modifiable risk factors in the CANRISK. These modifiable risk factors include weight loss, healthy diet with more fruits and vegetables and physical activity. An array of documents, resource contacts and tools were identified or developed. When available, copies of the documents were ordered in languages spoken by the target population. A two-page document on setting a healthy goal was designed by the team and translated into the various languages spoken by the participants. Participants were offered a fridge magnet plate showing healthy portion sizes and/or a pedometer with handbook on its use. HLP staff developed PowerPoint presentations on preDM and DM and their prevention for use in group sessions for study participants and others. These were then translated into Chinese, Vietnamese and Spanish in collaboration with community partners. As a follow-up to the study for research participants and others, HLP staff is offering several group session options to educate about preDM and its prevention. Study participant and health care provider satisfaction measurement VCH evaluation staff designed a seven-item outcome and satisfaction evaluation survey. Participants were requested to fill out this anonymous survey after they had completed the study. The evaluation survey asked about participants’ overall satisfaction with both parts of the study, namely, filling out the CANRISK and the blood test. That some participants would find the 2-hour test overly long and the glucose solution’s physiological effect uncomfortable was expected. A five-item satisfaction survey was emailed to eleven professionals from VCH and partner organizations. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Statistical analyses Table 1 Comparison of recruitment strategy outcome for CANRISK pilot study, Vancouver, Canada (N = 556) An additional variable was created in the dataset to denote ethnic group based on the origin of biological parents†. Only participants with both parents of the same ethnic origin were included in the analyses that examined differences among ethnic groups. We used SPSS version 14 for Windows (IBM) for all of our analyses. Recruitment strategy Results The Vancouver site surpassed its goal of enrolling 300 or more participants with 556 completing the study. Table 1 summarizes the most successful participant recruitment strategies. 4 VCH clinician referral 4 Partnership with UBC 6 VCHRI email to staff 16 Churches 17 Community organizations 26 Word of mouth from participants 27 Abbreviations: CANRISK, Canadian Diabetes Risk Assessment Questionnaire; UBC, University of British Columbia; VCH, Vancouver Coastal Health Authority; VCHRI, Vancouver Coastal Health Authority Research Institute. Table 2 Recruitment by biological ethnic group as compared to population in Vancouver in CANRISK pilot study, Vancouver, Canada (N = 571 600) Baseline characteristics Information about baseline characteristics of the ethnic groups in the sample appears in the following series of tables. Table 2 shows the ethnic composition of the study sample as compared to that of the City of Vancouver based on the 2006 Canadian census.5 In the study sample, the percentage of participants from three of the targeted ethnic groups exceeded their respective weight in the ethnic composition of the City of Vancouver. This was due to the Program’s strong connections with East Asian, South Asian and Latin American ethnic communities. Due to the different outreach strategies with ethnic communities, there are some marked differences in the characteristics of the sub-samples from these populations (Table 3). The Latin American sample consists of participants that are both younger than other ethnic groups (ANOVA: p < .001; then Tukey’s test: p < .01) and with a more equal gender distribution (Mann-Whitney test: p < .01) since over 60% were recruited from a university setting. On the other hand, South Asian participants are significantly older (p < .01) with 48% of participants in the 65- to 74-year age group and 86% women (p < .01). Recruitment of this ethnic group was largely through a community group targeting senior South Asian women. † Participants recruited, % Private practice physician referral Population group Study sample, n City of Vancouver5 % % East Asian 333 60 40.3 Caucasian 111 20 49.0 South Asian 50 9 5.7 Latin American 44 8 1.4 Other 18 3 3.6 Total 556 100 100.0 a Abbreviations: CANRISK, Canadian Diabetes Risk Assessment Questionnaire. a The “Other” category is not detailed separately in Tables 3, 5 and 6. Table 3 Age and sex by biological ethnic group recruited for CANRISK pilot study, Vancouver, Canada (N = 556) Variable Ethnic group, % All, % East Asian (n = 333) Caucasian (n = 111) South Asian (n = 50) Latin American (n = 44) (N = 556) 75 78 86 55 75 5 9 4 48 10 40–44 9 15 10 11 10 45–54 33 34 12 16 30 55–64 37 31 26 16 33 65–74 16 11 48 9 17 Sex Women Age group, years 30–39 Abbreviations: CANRISK, Canadian Diabetes Risk Assessment Questionnaire. Note: Only participants with both parents of the same ethnic origin were included in the analyses that examined differences among ethnic groups. CANRISK Q9 and Q10: Please check off which of the following ethnic groups your biological (blood) parents [mother, father] belong to: White (Caucasian); Aboriginal (First Nations person, Métis, Inuit); Black; Latin American; South Asian (East Indian, Pakistani, Sri Lankan, etc.); East Asian (Chinese, Vietnamese, Filipino, Korean, etc.); Other. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 14 Table 4 Number of CANRISK pilot study participants aged 30 to 39 years (n = 53) with risk factor for diabetes according to Canadian Diabetes Association, Vancouver, Canada Participants n % Parent or sibling with diabetes 15 28.0 Ethnicity: East/South Asian, Latin American, Aboriginal, sub-Saharan African 43 81.0 1 2.0 CDA diabetes risk factors History of large birth-weight baby (> 4 kg or 9 pounds) History of gestational diabetes 2 4.0 Presence of IGT or IFGa 4 7.5 6 11.0 Overweight or obesity (BMI ≥ 25 kg/m ) 20 38.0 Waist circumference above cut-off b 20 38.0 Hypertension 2 Total CDA proxy risk score = 0 5 9.5 Total CDA proxy risk scorec ≥ 1 48 90.5 Total CDA proxy risk scorec ≥ 2 32 60.0 c CANRISK score, points < 7 (low risk) 33 62.0 7–11 (slightly elevated risk) 17 32.0 12–14 (moderate risk) 2 4.0 15–20 (high risk) 1 2.0 > 20 (very high risk) 0 0 Abbreviations: BMI, body mass index; CANRISK, Canadian Diabetes Risk Assessment Questionnaire; CDA, Canadian Diabetes Association; IGT, impaired glucose tolerance; IFG, impaired fasting glucose. a Reporting having had a high blood sugar result in the past in CANRISK Q7‡. Used as proxy. b For women > 31.5 in/80 cm; for men > 37.0 in/94 cm. c A proxy risk score was calculated based on the presence or absence of the 8 CDA risk factors for which data are available in the CANRISK survey. No data are available in CANRISK on the CDA risk factors relating to: “high cholesterol or other fats in the blood” or to having “been diagnosed with any of the following conditions: polycystic ovary syndrome, acanthosis nigricans, schizophrenia. Table 5 Healthy living behaviours by biological ethnic group recruited for CANRISK pilot study, Vancouver, Canada Behaviour Ethnic group, % All, % East Asian (n = 333) Caucasian (n = 111) South Asian (n = 50) Latin American (n = 44) (N = 556) Every day 90 82 86 84 87 Not every day 10 18 14 16 13 Yes 60 55 82 48 60 No 40 45 18 52 40 Eat fruits and vegetables ≥ 30 min physical activity daily Abbreviations: CANRISK, Canadian Diabetes Risk Assessment Questionnaire. ‡ The CDA suggests that those with one or more diabetes risk factors6 be tested earlier than age 40 years. The CANRISK includes questions on eight factors from the CDA list of risk factors. An analysis of these risk factors in those aged 30 to 39 years was performed to review the appropriateness of including this age group in the study. Table 4 shows the number and percentage of participants presenting with each CDA risk factor. While 60% of those aged 30 to 39 years presented with two or more risk factors, the majority of participants in this age group (62%) were in the low risk CANRISK category. Four participants in this age group presented with either impaired fasting glucose (IFG) or impaired glucose tolerance (IGT). Their CANRISK scores were in the low risk (n = 2), slightly elevated risk (n = 1) and moderate risk (n = 1) categories. The pilot study team were concerned that the CANRISK question on fruit and vegetable consumption§ was not worded to include a minimum number of portions in order to obtain a zero risk point score. A comparison of the answers on this question and of the responses to the physical activity question appears in Table 5. Case detection (diabetes and prediabetes) An important objective of the screening was to provide an opportunity for earlier identification of people with DM and preDM through OGTT testing. Table 6 shows the laboratory testing results of the participant sample. Our study included participants who had been previously told that they had preDM (fasting plasma glucose [FPG]: 6.1–6.9 mmol/L), and 98 participants (18%) self-reported in the CANRISK that they had had a high blood sugar result‡. Of these, 26.5% had elevated results (IFG, IGT or both) while 7.1% were in the DM range. Alternately, among the 82% of participants who had never been told they had an abnormally high blood sugar, our study CANRISK Q7: Have you ever been found to have a high blood sugar (abnormal) either from a blood test, during an illness, or during pregnancy? Yes/No or don’t know. CANRISK Q5: How often do you eat vegetables or fruits? Every day/Not every day. § 15 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Table 6 Blood glucose range by biological ethnic group, CANRISK pilot study, Vancouver, Canada Result Category identified 15% as having either IFG or IGT or both, and 2% of these results were in the DM range. Ethnic group Alla Caucasian (n = 111) n % n % n % n % n % Normal 261 78.5 94 85.0 36 73.5 40 93.0 443 80.0 IFG only 12 3.5 2 2.0 3 6.0 1 2.5 18 3.0 IGT only 40 12.0 9 8.0 5 10.0 1 2.5 61 11.0 Both IFG and IGT 11 3.5 3 2.5 2 4.0 1 2.5 17 3.0 9 2.5 3 2.5 3 6.0 0 0.0 15 3.0 Diabetes range South Asian (n = 50) Evaluation of study participant and health care provider satisfaction measurement East Asian (n = 333) Latin American (n = 44) Abbreviations: CANRISK, Canadian Diabetes Risk Assessment Questionnaire; IFG, impaired fasting glucose; IGT, impaired glucose tolerance. a Missing laboratory test data for two participants. Figure 1 Overall satisfaction with the research process (CANRISK and blood test) in response to participant evaluation survey (n = 441), Vancouver, Canada a) Overall, my level of satisfaction with the research process to identify my risk of diabetes (blood test and CANRISK survey) is: 50% Percentage 40% 30% 20% 10% 0% Excellent Very Good Good Fair b) The CANRISK survey questions are clear and easy to understand. 50% 40% Percentage There was a 79% response rate to the participant evaluation survey, with 441 research participants responding. The results of the quantitative evaluation questions are illustrated in Figures 1a to 1c. In answer to the question of level of satisfaction with the research process, 25% of respondents rated this as good or fair (choices were fair, good, very good and excellent) (Figure 1a). These participants may have found the OGTT particularly uncomfortable (due to pain, bruising and swelling because of the venipuncture and nausea or dizziness from the glucose solution). In comparison, 96% either agreed or strongly agreed that the survey wording was clear and easy to understand (Figure 1b). Further, 94% of respondents either agreed or strongly agreed that they had received useful information about how to lower their risk of DM (Figure 1c). 30% 20% 10% 0% Strongly Agree Agree Neutral Disagree Strongly disagree c) The information I received about how to lower my risk of diabetes is really useful. In written comments about how to improve the CANRISK, several participants suggested that the question on blood relatives with DM** was confusing and that it was difficult to add up the risk score correctly. Others suggested providing an adjustment to the waist circumference question†† to include the target waist circumference interval suggested for Asians by the World Health Organization7 (90 cm versus 94 cm for Caucasians). Ten VCH and community group staff members who were involved in recruiting and supporting study participants filled out an evaluation survey, a response rate 50% Percentage 40% **CANRISK Q8: Have any of your blood relatives ever been diagnosed with diabetes? Select from: Mother; Father; Brothers/Sisters; Children; Other. 30% 20% †† 10% 0% Strongly Agree Agree Neutral Disagree Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Strongly disagree 16 CANRISK Q3: Men Waist circumference: Less than 94 cm or 37 inches/between 94–102 cm or 37–40 inches/ Over 102 cm or 40 inches; Women Waist circumference: Less than 80 cm or 31.5 inches/between 80–88 cm or 31.5–35 inches/Over 88 cm or 35 inches. Evaluation of lifestyle behaviour Most participants planned one or more changes in areas relating to the CANRISK questions or to accessing community resources suggested by the pilot study team, as shown in Figure 2. Only 5.7% of participants indicated that they were not thinking of adjusting their lifestyle. One professional also noted that some followed-up participants were actively making lifestyle changes as a result of participating in the study. Discussion The most effective strategy to recruit members of various ethnic groups was to partner with their community organizations and churches and then build on the resulting information exchange among members. For example, about 50% of East Asian participants were recruited through these channels. A few participants mentioned that they had been referred by their private practice physician. This could be due to patients not following through after the study brochure was handed-out by their physician. However, no requests for additional brochures were received from partner physicians. A successful strategy with Caucasians was the approach through the VCH Research Institute that circulated the study email to VCH staff. Several of the approximately 90 participants thus recruited then circulated the email to relatives and friends. ‡‡ Figure 2 Percentage of CANRISK pilot study participants thinking of making behaviour changes to lower their risk of diabetes, Vancouver, Canada 100% 80% Percentage of 91%. Rating their satisfaction with the CANRISK on a scale of excellent to poor, 70% rated it as very good, 10% as good and 20% as fair. Notably, the CANRISK was rated less highly by those working with low-income immigrant communities. They noted that the survey was too long for people with low literacy levels. It was also suggested that the wording regarding ethnic groups be reviewed (e.g. replace such words as “Black” and “White”). One community partner and all the VCH professionals are planning to continue using the CANRISK in their practice. 60% 40% 20% 0% Eat more fruits/veggie + Physical activity In the overall sample and in most ethnic groups, substantially more participants ate one or more fruits or vegetables every day‡‡ compared to being physically active for 30 minutes every day§§. Recommended targets on fruit and vegetable consumption in healthy living initiatives usually start at 5 or more portions per day.8 The Vancouver team suggests that the CANRISK question should be amended to mention the higher fruit and vegetable targets in accordance with the 7 to 10 daily portions recommended for adults by the Canada Food Guide.9 This would improve the usability of the CANRISK as a teaching and awareness-raising tool. None of the participants who scored in the DM range knew of their health status prior to enrolling in the study. All gave the name of their physician, to whom the team then sent a letter with their test results. They were also referred to a Diabetes Education Centre, including the Chinese Diabetes Education Centre for Chinese speakers. A Vancouver site success is that participants with abnormal laboratory test results, who were subsequently diagnosed by their physician as having DM or preDM, were provided with timely educational interventions in their own language and linkages to community resources to support them in their self-management efforts. Due to the significant differences in age stratification and the unequal numbers in the ethnic subgroups, it is not appropriate to comment on the levels of preDM and DM detected across ethnic groups. Lose weight Access resources In terms of the cost-benefit of testing all participants with an OGTT rather than targeting those with an FPG equal or greater than 6.1 mmol/L as recommended by the CDA,4 we identified 61 participants (11% of the study participants) who had an isolated IGT who would not have been detected by FPG screening. Conclusion Overall, the recruitment and screening process was successful in the targeted ethnic communities. It resulted in identifying 15 participants (3%) with test results in the DM range, while 96 participants (17%) had results in the preDM range. Among these, 11% had IGT only which would not have been detected using only an FPG test. It was essential to use multiple approaches for participant recruitment in order to enrol participants from the varied ethnic communities in Vancouver. Once a minimum number of individuals from a particular ethnic community had been recruited, word-of-mouth snowballed more referrals. The team is reviewing strategies to further engage with primary care physicians to increase the number of patient referrals to VCH health promotion and diabetes prevention programming. Ongoing discussions are underway about how best to integrate the CANRISK in these different primary care clinic environments based on their specific ways of working. CANRISK Q5: How often do you eat vegetables or fruits? Every day/Not every day. CANRISK Q4: Do you usually do some physical activity such as brisk walking for at least 30 minutes every day? This activity can be done while at work or at home. Yes/No. §§ 17 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada The research partnership between PHAC and HLP created synergies and furthered the program’s aims. The team has formed new alliances with ethnic community leaders and groups to promote healthy living habits, increase awareness of DM risk factors and develop culturally appropriate content in several languages. The CANRISK provides an important basis for screening and teaching regarding the three pillars that are HLP’s focus: healthy eating, increasing activity levels and smoking cessation. 8. Agence de la santé et des services sociaux de la Capitale-Nationale Quebec. 0-5-30 combinaison prevention [Internet]. [Quebec City]: Province of Quebec; [cited 2011 Jul 03]. Available from: http://www.0-5-30.com 9. Health Canada. Easting well with Canada’s food guide. Ottawa (ON): Health Canada; [modified 2011 Mar 7; cited 2011 Jul 03]. Available from: http://www.hc-sc.gc.ca /fn-an/food-guide-aliment/index-eng.php References 1. Kaczorowski, J, Robinson, C, Nerenberg, K. Development of the CANRISK questionnaire to screen for prediabetes and undiagnosed type 2 diabetes. Can J Diabetes. 2009;33(4):381-5. 2. Odegaard AO, Koh WF, Vasquez G, Arakawa K, Lee H-P, Yu MC, et al. BMI and diabetes risk in Singaporean Chinese. Diabetes Care. 2009;32(6):1104-6. 3. Ramachandran A, Snehalatha C, Kapur A, Vijay V, Mohan V, Das AK, et al. High prevalence of diabetes and impaired glucose tolerance in India: National Urban Diabetes Survey. Diabetologia. 2001;44:1094-101. 4. Canadian Diabetes Association. 2008 clinical practice guidelines for the prevention and management of diabetes in Canada. Can J Diabetes. 2008; 32 (9): S15. 5. Statistics Canada. Vancouver, British Columbia. 2006 Community profiles: visible minority population characteristics. Ottawa (ON): 2007 [update 2010 Dec 06; cited 2011 Jul 12]. [Statistics Canada, Catalogue No.: 92-591-XWE]. Available from: http://www12.statcan.ca/census-recensement /2006/dp-pd/prof/92-591/index.cfm?Lang=E 6. Canadian Diabetes Association. Diabetes facts: What are the risk factors for diabetes? [Internet]. Toronto (ON): Canadian Diabetes Association; Dec 2009 [cited 2011 Jul 03]. Available from: http://www.diabetes.ca /files/Diabetes_Fact_Sheet.pdf 7. Low S, Chin MC, Ma S, Heng D, Deurenberg-Yap M. Rationale for redefining obesity in Asians. Ann Acad Med Singapore. 2009;38(1):66-9. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 18 Validating the CANRISK prognostic model for assessing diabetes risk in Canada’s multi-ethnic population C. A. Robinson, MA (1); G. Agarwal, MBBS (2); K. Nerenberg, MD, MSc (3) This paper has been peer reviewed. Abstract Introduction: Despite high rates of undiagnosed diabetes and prediabetes, suitable risk assessment tools for estimating personal diabetes risk in Canada are currently lacking. Methods: We conducted a cross-sectional screening study that evaluated the accuracy and discrimination of the new Canadian Diabetes Risk Assessment Questionnaire (CANRISK) for detecting diabetes and prediabetes (dysglycemia) in 6223 adults of various ethnicities. All participants had their glycemic status confirmed with the oral glucose tolerance test (OGTT). We developed electronic and paper-based CANRISK scores using logistic regression, and then validated them against reference standard blood tests using test-set methods. We used area under the curve (AUC) summary statistics from receiver operating characteristic (ROC) analyses to compare CANRISK with other alternative risk-scoring models in terms of their ability to discern true dysglycemia. Results: The AUC for electronic and paper-based CANRISK scores were 0.75 (95% CI: 0.73–0.78) and 0.75 (95% CI: 0.73–0.78) respectively, as compared with 0.66 (95% CI: 0.63–0.69) for the Finnish FINDRISC score and 0.69 (95% CI: 0.66–0.72) for a simple Obesity model that included age, BMI, waist circumference and sex. Conclusion: CANRISK is a statistically valid tool that may be suitable for assessing diabetes risk in Canada’s multi-ethnic population. CANRISK was significantly more accurate than both the FINDRISC score and the simple Obesity model. Keywords: diabetes, prediabetes, screening, risk assessment, FINDRISC, blood sugar, public health Introduction Despite high rates of undiagnosed diabetes and prediabetes in Canada, the assessment tools currently used to estimate an individual’s risk of diabetes are lacking. It is clinically important to be able to identify individuals at risk for diabetes. First, undiagnosed diabetes often remains undetected for 4 to 7 years before clinical diagnosis, and many newly diagnosed patients already exhibit signs of microvascular and macrovascular complications.1,2 Second, individuals with prediabetes (impaired fasting glucose [IFG] and/or impaired glucose tolerance [IGT]) have a high likelihood of developing type 2 diabetes—10 to 20 times that of normo glycemic persons.3,4 As such, adults with prediabetes are the most likely to benefit from early interventions.3,4 Large randomized experimental studies such as the Finnish Diabetes Prevention Study5 and the US Diabetes Prevention Program6 have demonstrated that lifestyle intervention can effectively reduce the incidence of diabetes among those with prediabetes. Risk-scoring questionnaires may be useful to enhance individual risk assessment and lifestyle education. They could also lead to more cost-effective diabetes screening approaches. Several prognostic risk-scoring models for type 2 diabetes are currently available for clinical use.7-14 However, most require specific blood test results, which presumes that a clinical encounter or diagnostic testing has already taken place. This limits widespread use of these models from a public health perspective. A diabetes risk assessment approach that relies only upon information a participant can self-complete without detailed knowledge of specific laboratory test values has been developed in Finland. The Finnish Diabetes Risk Score15 (FINDRISC) is a key element of Finland’s national FIN-D2D diabetes prevention program, which has successfully screened over 10% of the Finnish population so far. FINDRISC has been used in Finland to identify high-risk individuals who might benefit from interventions or who would merit further investigation using the oral glucose tolerance test (OGTT). Among those detected by the Finnish study as being at high risk of developing diabetes, 60% of men and 45% of women already had abnormal glucose tolerance at baseline.16 The incidence of diabetes at one-year follow-up was between 18% and 22% among those who had high-risk prediabetes (i.e. both IFG and IGT) at baseline. Of those who completed a lifestyle education program, 17% reduced their body weight by Author references: 1. Centre for Chronic Disease Prevention and Control, Public Health Agency of Canada, Ottawa, Ontario, Canada 2. Department of Family Medicine, McMaster University, Hamilton, Ontario, Canada 3. Department of Medicine, Royal Alexandra Hospital, Edmonton, Alberta, Canada Correspondence: Chris Robinson, Centre for Chronic Disease Prevention and Control, Public Health Agency of Canada, 785 Carling Avenue, Ottawa, ON K1A 0K9; Tel.: (613) 957-9874; Fax: (613) 941-2633; Email: [email protected] 19 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada over 5%; as a result, their risk of developing diabetes was 69% lower than that of those with stable weight.17 However, the generalizability of FINDRISC is limited by the different ethnic make-up of Canada compared to that of Finland. As a result, Canadian diabetes experts adapted FINDRISC to include ethnicity and other key variables (sex, education, macrosomia) to create the Canadian Diabetes Risk Assessment Questionnaire (CANRISK).18,* This paper describes three main objectives of our study: (1) to develop a risk-scoring prognostic model (similar to FINDRISC score) suitable for Canada’s multi-ethnic population (CANRISK); (2) to validate the resulting scoring model using a test-set methodology to assess dysglycemia from measured blood tests; and (3) to compare the predictive accuracy of the new CANRISK model to FINDRISC. Methods Data source Between 2007 and 2011, 6475 Canadian adults from seven provinces (British Columbia, Saskatchewan, Manitoba, Ontario, New Brunswick, Nova Scotia and Prince Edward Island) were recruited in a screening study to detect diabetes and prediabetes using the CANRISK questionnaire. Several large urban sites were deliberately included to ensure a diverse multi-ethnic sample of participants. All participants had their glycemic status confirmed with the oral glucose tolerance test (OGTT, i.e. fasting plasma glucose [FPG] and plasma glucose 2 hours after a 75 g glucose challenge). A subset of participants at three CANRISK sites also had their glycated hemoglobin (HbA1c) measured. Most participants were recruited through face-to-face encounters during opportunistic visits at community health centres;19 some were recruited through local mailouts.20 Most participants were aged 40 to 74 years, although some sites chose to include younger Aboriginal participants and those from other non-White ethnic groups. Eligibility criteria for inclusion in the study included the following: no previous diagnosis of diabetes (or prediabetes at some pilot sites); not currently pregnant; able to complete the CANRISK questionnaire in English or French, with assistance if required (most sites, although other language versions were also available at several urban pilot sites); not currently using metformin or other glucose-modifying prescription drugs (some pilot sites); and living within the local study area. Data restrictions (core data) For estimating the various prognostic models we restricted the CANRISK dataset to those participants who had complete data for key variables (blood test results, age, sex, ethni city, height, weight). We imputed missing waist circumference (6% of core cases) from mean values obtained from participants with valid data, stratified by age, sex, and body mass index (BMI) (see Table 1). Missing family history was also imputed (i.e. assumed to be “no” for 13% of core cases). Cases with item-missing data for other variables were dropped from the final regression models. Predictor variables We derived certain predictor variables from answers to the CANRISK questionnaire (e.g. BMI from weight and height). We converted continuous variables such as age and BMI into categorical variables and then adopted a dummy variable approach for logistic regression analysis. This allowed nonlinearities in the predictor variables while still generating a practical scoring algorithm where scores can be summed using simple arithmetic (e.g. the paper-based version of the CANRISK scoring tool). Smoking status was only available for selected pilot sites (63% of total observations) since this question was added to the CANRISK questionnaire during the last phase of data collection. (The smoking variable was intended for use in other potential data linkage studies regarding cardiovascular risk. For this reason, and because of the large percentage of item-missing data, smoking was not included as a predictor in the CANRISK dysglycemia prognostic model.) * http://www.diabetes.ca/documents/for-professionals/NBI-CANRISK.pdf. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 20 Outcome variable For the purposes of validation, the outcome for the prognostic model was dysglycemia based on the collective results of participants’ blood tests (FPG and 2-hour 75 g OGTT value) according to standard World Health Organization 2006 criteria.21,22 Model validation and performance: general approach Following standard statistical methods, we validated the CANRISK model using the split-sample test-set approach.23 This process of internal validation involved randomly splitting the core CANRISK dataset into a derivation “test” dataset made up of 70% of the available cases (n = 4366), with the remaining 30% “set” data (n = 1857) serving as the validation dataset. In the first step, we used the “test” training data to estimate the prognostic model using logistic regression. The Hosmer-Lemeshow summary statistic and the associated Brier score24 were used to assess the goodness-of-fit of the model. We then used the resulting regression coefficients to predict dysglycemia in the “set” dataset. We assessed the accuracy of the regression model (i.e. discrimination in terms of correctly classifying true-positive cases with dysglycemia) using receiver operating characteristic (ROC) curves. For measuring the overall performance of the regression model in terms of predictive validity, we used the area under the curve (AUC) summary statistic (i.e. the concordance c statistic). Finally, for various potential CANRISK score thresholds, we calculated standard measures of sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) in order to assess the diagnostic validity of the screening test at each threshold. Creating the CANRISK prognostic model for dysglycemia As the first step, we used data from the cross-sectional test subsample to estimate three logistic regression models to predict the dysglycemia outcome. These were Table 1 Characteristics of core CANRISK participants (n = 6223) Q Characteristics by response to CANRISK questionsa 3 1 Male Age, years (mean = 52.6; SD = 12.5) 19–44 45–54 55–64 65–78 BMI (kg/m2)b Normal/underweight (< 25) Overweight (25–29.9) Obese, non-morbid (30–34.9) Obese, morbid (35+) Waist circumference (cm) Male < 94 / Female < 80 Male 94–102 / Female 80–88 Male > 102 / Female > 88 Daily brisk physical activity ≥ 30 minutes No Daily consumption of fruit/vegetables No High blood pressure diagnosed by a doctor or nurse / has taken medication for blood pressure Yes High blood sugar confirmed by a blood test / during an illness / during pregnancy Yes Positive family history of diabetesd Mother Father Sibling Child Other relatives Ethnicity (mother) White (Caucasian) Aboriginal Black Latin American South Asian East Asian Other Ethnicity (father) White (Caucasian) Aboriginal Black Latin American South Asian East Asian Other Education Some high school or less High school diploma Some college or university University or college degree 2 3 4 5 6 7 8 9 10 11 21 Percentage, % 36.4 Valid number, n 2263 Number with missing data 0 0 26.4 27.5 28.5 17.6 1644 1712 1774 1093 42.8 33.0 15.8 8.4 2666 2052 982 523 19.5 26.4 54.1 1213 1643 3367 37.8 2350 13 23.9 1484 4 31.6 1954 46 13.5 822 141 25.7 20.2 24.6 2.5 33.2 1390 1039 1301 148 1795 824 1077 933 326 824 65.7 12.1 3.5 2.8 5.3 10.1 1.0 4089 756 220 175 328 629 63 0 0 0 0 0 0 0 66.0 11.3 3.6 2.7 5.3 10.2 1.2 4084 698 222 169 327 632 72 34 31 31 30 30 30 34 16 23.2 21.4 26.8 28.6 1443 1330 1669 1781 Continued on the following page 0 368c Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Table 1 (Continued) Characteristics of core CANRISK participants (n = 6223) Q Characteristics by response to CANRISK questionsa 12 Self-rated health status Excellent Very good Good Fair/poor Smoking statuse Daily cigarettes History of gestational diabetes (% females) History of macrosomia (% females) 13 15 16 Percentage, % Valid number, n 10.4 33.2 42.1 14.3 648 2067 2618 890 13.6 7.5 22.0 534 258 678 Number with missing data 27 2294 268 202 Abbreviations: BMI, body mass index; CANRISK, Canadian Diabetes Risk Assessment Questionnaire; Q, question number from CANRISK. a For the complete version of the CANRISK questions, see http://www.diabetes.ca/documents/for-professionals/NBI-CANRISK.pdf. b From self-reported weight and height. c Imputed missing waist circumference (6% of core cases) from mean values obtained from participants with valid data. d Missing family history (13% of core cases) was assumed to be “no”. e These responses come from selected pilot sites only. (1) the Obesity model, using BMI, waist circumference, age and sex. (This basic model was intended to reflect observable risk factors commonly used for diabetes screening); (2) the FINDRISC Variables model, using the eight questions in FINDRISC (i.e. the first eight questions on CANRISK). (This model reflected how well the FINDRISC variables predicted dysglycemia in a cross-sectional analysis within the CANRISK dataset); and (3) the CANRISK model, using all the variables available from the CANRISK questionnaire. (This “full information” model reflected ethnicity and other variables added to the basic FINDRISC Variables model). Statistical analysis In developing the CANRISK prognostic model we recognized that the existing FINDRISC scores derived from 10-year cumulative incidence (i.e. definitive long-term diabetes outcome) should be retained and enhanced, rather than replaced with an entirely new prognostic model based on current dysglycemia (i.e. short-term risk condition from blood testing on one occasion). Our statistical methods therefore reflect our analytical objective to adapt the existing FINDRISC prognostic model by including ethnicity and other key variables to ensure generalizability to the Canadian population. Minimizing the number of predictor variables was not paramount in this case. Using the “test” training dataset, we proceeded to develop the CANRISK prognostic model according to the following steps: (1)We assessed correlations between the dependent variable (dysglycemia) and various independent variables (pre dictors). We also assessed correlations between predictors to identify potential multicollinearity, which would violate the independent variable assumption. (2)We conducted univariate analyses to determine the strength of association between dysglycemia and individual predictors. We used these results to determine the order of entry of the Canadian predictors into the CANRISK model. (3)We forced FINDRISC’s eight questions into a logistic regression to create the FINDRISC Variables model, measuring its performance in terms of goodness-of-fit and accuracy. (4)We added ethnicity and other potential predictors to the basic FINDRISC Variables model in a series of steps, assessing gains in model performance at each step, and using the likelihood ratio to assess the added predictive power. Variable selection in the final CANRISK prognostic model therefore Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 22 involved maximizing the correct classification of true-positive cases by the overall model, while ensuring goodness-of-fit as well as statistical significance of the overall model and individual predictors at α = 0.05. Each variable in the final CANRISK model was also subject to a priori expectations regarding the correct sign, meaning that a known risk factor should have a positive coefficient and a known protective factor should be negative. Statistical analyses were performed using SPSS version 15.0 for Windows.25 Results The study population Figure 1 illustrates how the available data were organized for analysis. We excluded 3.9% of participants with missing data for key variables from the “core” dataset. Table 1 describes ethnicity and other key characteristics of the 6223 persons remaining in the core dataset and related item-missing data for individual variables. Blood test results (Table 2) showed that 20.5% of the participants tested positive for dysglycemia (15.7% prediabetes; 4.8% newly detected diabetes). Of the 1273 cases of dysglycemia identified, only 545 (43%) would have been identified using fasting glucose alone. Figure 1 CANRISK data Total CANRISK observations (N = 6475) Obesity model (n = 4366) 100% test Core data Not missing key data (n = 6223) 96% total Non-core data Missing key data (n = 252) 4% total “Test” data (n = 4366) 70% core “Set” data (n = 1857) 30% core FINDRISC Variables (n = 4251) 97% test CANRISK model (n = 4091) 94% test Estimation of the CANRISK prognostic model Table 3 presents the three different prognostic models that we estimated using logistic regression methods applied to the core CANRISK data. In terms of goodness-of-fit and overall significance, all three models were highly significant based on likelihood ratio and Pearson chi-square (χ2) at p < .001. The HosmerLemeshow summary statistic also indicated that each of the models was a good fit. The Brier score24 for the CANRISK prognostic model was 0.002; the typical range is 0 (perfect) to 0.25 (no predictive value). The resulting CANRISK prognostic model includes several key risk factors—notably ethnicity—as well as family history, waist circumference, BMI and other key variables. As indicated by the odds ratios (ORs) in Table 3, non-White ethnicity was a signi ficant risk factor compared to the White reference group (e.g. OR = 2.69 for South Asian people; 2.61 for East Asian people; 1.35 for Aboriginal people). Black ethnicity (OR = 1.53; 95% CI: 0.92–2.54) was not statistically significant but showed the correct sign (positive coefficient) and was plausible based on other epidemiological studies;26-28 it was therefore retained. Latin American ethnicity and Other ethnicity were both statistically insignificant. Compared to high educational attainment at the university or college level, low educational attainment (OR = 1.60 for less than high school) was statistically significant as a risk factor, although having only a high school diploma was not. We retained the latter to reflect the increasing risk associated with patterns of low education. Being male (OR = 1.68) was another significant risk factor in the CANRISK model. (It was excluded from the original FINDRISC model). Compared to no family history of diabetes, positive family history (i.e. OR = 1.21 for the number of categories of first-degree relatives affected with diabetes: mother, father, sibling, child) was also significant in the CANRISK model (family history of diabetes had not been directly estimated in FINDRISC). Family history for seconddegree relatives was statistically insignificant and had the wrong sign (negative coefficient), and was therefore rejected. Diet and physical activity variables were not statistically significant but did generate the correct a priori sign (positive coefficient). In keeping with the FINDRISC approach, we retained these lifestyle variables in the model for educational purposes. For similar reasons, we also retained macro somia (i.e. women who gave birth to a child weighing 4.1 kg or more) in the CANRISK model despite its statistical insignificance. Other potential variables such as self-reported health status were tried but rejected due to implausible sign and statistical insignificance of the coefficient. Two variables were dropped due to multicollinearity: history of gestational diabetes was highly correlated with history of high blood sugar, and father’s ethnicity Table 2 Blood test results used for validating CANRISK prognostic model Blood test resultsa Percentage of total,b,c % Cases detected, n A Isolated IFG 3.8 238 B Isolated IGT 9.2 573 C High-risk prediabetes (IFG and IGT) D Total cases of prediabetes = A + B + C 2.6 163 15.7 974 E Diabetes detected via FPG only 0.8 52 F Diabetes detected via OGTT glucose challenge only 2.5 155 G Diabetes detected via both FPG and OGTT glucose challenge 1.5 92 H Total cases of screen-detected diabetes = E + F + G Total cases of dysglycemia = D + H Cases with HbA1c > 6.5% from subset of 1057 participantsd 4.8 299 20.5 1273 4.2 44 Abbreviations: FPG, fasting plasma glucose; HbA1c, glycated haemoglobin; IFG, impaired fasting glucose; IGT, impaired glucose tolerance; OGTT, 2-hour 75 g oral glucose tolerance test. a Results are based on standard 2006 World Health Organization diagnostic criteria.15,16 b n = 6223 participants in the core dataset. c Values may not add up the total due to rounding. d Only selected pilot sites measured HbA1c. 23 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Table 3 Comparison of three estimated logistic regression models based on outcome of dysglycemia Logistic regression model CANRISK (n = 4091 test obs) FINDRISC Variablesb (n = 4251 test obs) Obesityc (n = 4366 test obs) 852 873 902 a Number of dysglycemia events in each model subsample, n OR 95% CI eCANRISK score (β)* pCANRISKd score OR 95% CI −3.84 Intercept β coefficient* OR 95% CI −3.31 β coefficient* 3.25 Variable Age, years 19–44 (ref) 1.00 45–54 2.01 1.00 1.53–2.63 0.70 7 1.77 1.00 1.37–2.28 0.57 1.98 1.55–2.52 0.68 55–64 3.33 2.55–4.37 1.20 13 2.81 2.20–3.59 1.03 3.27 2.59–4.13 1.19 65–78 4.21 3.12–5.69 1.44 15 3.65 2.78–4.79 1.29 4.33 3.37–5.57 1.47 BMI, kg/m2 < 25 (ref) 1.00 25–29.9 1.43 1.10–1.86 0.36 4 1.43 1.00 1.12–1.83 0.36 1.29 1.00 1.01–1.64 0.25 30–34.9e 2.43 1.78–3.33 0.89 9 2.74 2.07–3.63 1.01 2.12 1.59–2.82 0.75 35+ 3.70 2.61–5.24 1.31 14 3.55 2.60–4.84 1.27 Waist circumference, cm M < 94 / F < 80 (ref) 1.00 1.00 1.00 M 94–102/ F 80–88 1.51 1.11–2.06 0.41 4 1.27 0.94–1.70 0.24 1.46 1.10–1.95 0.38 M >102 / F > 88 1.74 1.24–2.45 0.56 6 1.29 0.95–1.76 0.26 1.77 1.30–2.42 0.57 0.94–1.33 0.11 1 0.92–1.29 0.09 0.95–1.43 0.15 2 1.30 1.07–1.57 0.26 1.20–1.70 0.36 4 1.42 1.20–1.68 0.35 3.14–4.79 1.36 14 3.72 3.04–4.55 1.31 1.32–1.84 0.44 Physical activity ≥ 30 min/day Yes (ref) 1.00 Nof 1.12 1.00 1.09 Eats fruit/vegetables every day Yes (ref) 1.00 Nof 1.16 1.00 History of high blood pressure No (ref) 1.00 Yes 1.43 1.00 History of high blood glucose No (ref) 1.00 Yes 3.88 1.00 Family history of diabetes None (ref) 1.00 First-degree relative with DMg 1.21 1.09–1.34 0.19 2 1.31 1.11–1.54 0.27 — — — — 0.74 0.61–0.89 −0.31 1.39–2.04 0.52 6 Any second degree relative affectedh 1.00 Sex Female (ref) 1.00 Male 1.68 1.00 1.56 Ethnicity White (ref) 1.00 Aboriginal 1.35 1.004–1.82 0.30 3 Blacki 1.53 0.92–2.54 0.43 5 East Asian 2.61 1.93–3.52 0.96 10 South Asian 2.69 1.90–3.82 0.99 11 Continued on the following page Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 24 Table 3 (Continued) Comparison of three estimated logistic regression models based on outcome of dysglycemia Logistic regression model CANRISKa (n = 4091 test obs) FINDRISC Variablesb (n = 4251 test obs) Obesityc (n = 4366 test obs) 852 873 902 Number of dysglycemia events in each model subsample, n OR 95% CI eCANRISK score (β)* pCANRISKd score OR 95% CI −3.84 Intercept β coefficient* OR −3.31 95% CI β coefficient* 3.25 Variable Macrosomia (women)f No or N/A (ref) 1.00 Yes 1.06 0.81–1.39 0.06 1 Education Some college/university (ref) 1.00 High school diplomaj 1.13 0.91–1.40 0.12 1 Less than high school 1.60 1.31–1.96 0.47 5 Abbreviations: BMI, body mass index; CANRISK, Canadian Diabetes Risk Assessment Questionnaire; CI, confidence interval; DM, diabetes mellitus; eCANRISK, electronic-based CANRISK score; F, female; FINDRISC, Finnish Diabetes Risk Score; M, male; N/A, not applicable; obs, observations; OR, odds ratio; pCANRISK, paper-based CANRISK score; ref, reference. Notes: Shaded cells in FINDRISC Variables and Obesity models were not part of the assessment. a Uses all the variables available from the CANRISK questionnaire (http://www.diabetes.ca/documents/for-professionals/NBI-CANRISK.pdf). b Uses the eight questions in FINDRISC (i.e. the first eight questions on CANRISK) and reflects how well the FINDRISC variables predicted dysglycemia in a cross-sectional analysis within the CANRISK dataset. c Uses BMI, waist circumference, age and sex to reflect observable risk factors commonly used for diabetes screening. d Maximum pCANRISK score is 81 for females, 86 for males. e In the FINDRISC Variables model, this group is combined with BMI ≥ 35 to represent body mass index of 30+ (i.e. similar to FINDRISC score variables). f Not statistically significant but retained in the model for educational purposes. g In the CANRISK model, this group counts the number of categories of first-degree relatives affected, while in the FINDRISC model this group indicates whether any first-degree relative was affected. h Statistically insignificant in the CANRISK model and with the wrong sign (negative coefficient). i Black ethnicity was not statistically significant but showed the correct sign (positive coefficient) and was plausible based on other epidemiological studies,29-31 and was therefore retained. j Having a high school diploma was not statistically significant but it was retained to reflect the increasing risk associated with patterns of low education. * p < .05 was highly correlated (0.92) with mother’s ethnicity. Including these variables in the model led to counterintuitive signs on the coefficients and decreased the goodnessof-fit in the model. (Note that this does not mean that father’s ethnicity is unimportant or should not be measured. Rather, it means that mother’s ethnicity can serve as a proxy measure for both parents when estimating the relevant model coefficient.) Electronic and paper-based CANRISK scores In order to implement the CANRISK model, specific threshold scores are required as potential credible cut-offs for determining broad categories of diabetes risk: low, medium and high. Because CANRISK scores may be applied in various public health and primary care settings, the scores have been calculated for two different formats: (1) a detailed “electronic” format (eCANRISK) suitable for programmed risk calculators (e.g. iPad App, online web calculator) and (2) a “paper-based” format (pCANRISK) based on simple arithmetic and rounded coefficients (such as FINDRISC). For the detailed electronic version, we calculated eCANRISK scores by summing the relevant beta coefficients from the logistic equation in Table 3 for applicable variables. For example, a 58-year-old White man with no other risk factors except for his mother having diabetes would have an eCANRISK score calculated as: −3.84 (intercept) + 1.20 (aged 55–64 years) + 0.52 (male) + 0.19 (multiplied by 1, since only one category of first-degree relative was affected with diabetes) + 0.00 (normal BMI, waist, etc.) = −1.93. 25 For the pCANRISK score, we followed the approach used by Sullivan et al.29 The score was calculated based on a rescaled, rounded version of the detailed beta coefficients that make up the eCANRISK score. The basic eCANRISK values were rescaled using the formula beta/0.09393 to total a maximum of 81 points for women and 86 points for men. Rescaling to a larger number was intended to minimize the effect of rounding error on the paper-based scores. Using the same example of a 58-year-old White man with no other risk factors except for his mother having diabetes the pCANRISK score would be calculated as: 13 (aged 55–64 years) + 6 (male) + 2 (multiplied by 1, since only one category of first-degree relative was affected with diabetes) = 21. This is low compared with the median paperbased pCANRISK score (28) for the entire Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Figure 2 Dysglycemia by CANRISK decile study population. (See Appendix A for a detailed explanation of how electronic and paper-based CANRISK scores may be used to estimate the probability of dysglycemia.) 60% Figure 2 conveys the complex risk factor relationships underlying the CANRISK score and illustrates the strong positive relationship between CANRISK score and true dysglycemia outcome, where dysglycemia prevalence in the highest CANRISK decile (57%) is 25 times higher than in the lowest decile (2%). 50% Prevalence 40% 20% 10% Assessing CANRISK’s overall performance: validating the model. We created CANRISK scores using the “test” training data, which were then applied using ROC analysis against the evaluation “set” dataset in order to validate the CANRISK logistic model against reference standard blood tests (FPG and 2-hour glucose challenge). This ROC analysis evaluated how well CANRISK is able to predict true dysglycemia (i.e. discrimination of true-positive and negative cases). As shown in Table 4, the discriminating power of each CANRISK model across the full range of possible risk score cut-offs is indicated by the AUC summary statistic. (This is also illustrated graphically by the ROC curve in Figure 3.) Based on the 30% validation “set” data, the AUC for eCANRISK and pCANRISK were both 0.75. 30% 0% 10 20 30 40 50 60 70 80 90 100 70 34 22% 3% 80 38 20% 7% 90 43 29% 7% 100 69 41% 16% CANRISK decile Decile Score Prediabetes Diabetes 10 14 2% 0% 20 18 4% 1% 30 22 5% 1% 40 25 9% 3% 50 28 12% 2% 60 31 16% 4% Table 4 AUC results for ROC curve analyses Model Validation "set" data (n = 1676) AUC 95% CI Electronic score (eCANRISK) 0.75 0.73–0.78 Paper-based score (pCANRISK) 0.75 0.73–0.78 FINDRISC Variables 0.73 0.70–0.76 Obesity model 0.69 0.66–0.72 FINDRISC score 0.66 0.63–0.69 Abbreviations: AUC, area under the curve; CANRISK, Canadian Diabetes Risk Assessment Questionnaire; CI, confidence interval; FINDRISC, Finnish Diabetes Risk Score; ROC, receiver operating characteristic. Comparing CANRISK and FINDRISC scores Finally, we established the diagnostic validity of pCANRISK as a potential screening test using selected scoring thresholds for detecting dysglycemia in the validation dataset (Table 5). These Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Figure 3 ROC curves 1.0 0.8 Sensitivity As shown in Table 4 and Figure 3, the ROC results compare the performance of various models in terms of their ability to accurately detect true dysglycemia. AUC results indicate that both the pCANRISK (0.75) and eCANRISK scores (0.75) are significantly more accurate than the FINDRISC score (0.66) and the simple Obesity model (0.69) to greater than 95% confidence level. CANRISK appears to be slightly more accurate than the FINDRISC Variables model though their confidence intervals overlap. 0.6 Source of the Curve 0.4 eCANRISK pCANRISK FINDRISC Variables Obesity Model FINDRISC Score 0.2 0.0 0.0 0.2 0.4 0.6 0.8 Specificity Diagonal segments are produced by ties. 26 1.0 Table 5 Predictive accuracy of CANRISK model at various scoring thresholds pCANRISK score Threshold score Sensitivity (detecting true dysglycemia) Specificity False-positive rate (1−specificity) PPV NPV Percent of total CANRISK participants with scores below threshold score (screened out), % 21 Slightly elevated 0.95 0.28 0.72 0.25 0.96 25 29 Moderate 0.80 0.55 0.45 0.31 0.92 50 32 Balanced 0.70 0.67 0.33 0.35 0.90 61 33 High 0.66 0.70 0.30 0.36 0.89 64 43 Very high 0.30 0.94 0.06 0.55 0.84 89 Abbreviations: CANRISK, Canadian Diabetes Risk Assessment Questionnaire; FINDRISC, Finnish Diabetes Risk Score; NPV, negative predictive value; pCANRISK, paper-based CANRISK; PPV, positive predictive value. Discussion Model building The CANRISK model includes terms for age, BMI, waist circumference, physical activity, fruit/vegetable consumption, history of high blood pressure, history of high blood glucose, family history of diabetes, sex, ethnicity, maternal history of macrosomia, and education. Four of these terms (sex, ethnicity, macrosomia and education) were 70 60 High 95% CI Mean 50 Low 95% CI 40 30 20 10 > 20: very high (1/2) 15–20: high risk (1/3) 12–14.9: moderate risk (1/6) 0 7–11.9: slightly elevated (1/25) Figure 4 illustrates the relationship between CANRISK and FINDRISC scores. For slightly elevated, moderate-risk, high-risk and very high-risk categories, diabetes over the next 10 years, as compared with 1 in 6 for those with moderate-risk scores and 1 in 25 for slightly elevated-risk scores. Figure 4 pCANRISK score by FINDRISC category Under 7: low risk (1/100) Table 5 shows the performance of pCANRISK at these five selected screening thresholds. (Note that these are arbitrary and do not necessarily indicate desirable screening thresholds). For a relatively low score equating with FINDRISC’s “slightly elevated” threshold, a pCANRISK score of 21 or higher would have sensitivity of 95% and specificity of 28% (72% false-positive rate). The positive predictive values (PPV) and negative predictive values (NPV) for this threshold would be 25% and 96% respectively. At the other extreme, restricting screening to those with a score of 43 or higher (i.e. FINDRISC’s “very high-risk” threshold) would markedly increase specificity and the proportion of CANRISK participants who would be screened out (for whom follow-up testing or intensive educational intervention would not be recommended), but would substantially decrease sensitivity and NPV. At the balanced cut-off score of 32, the sensitivity would be 70%, specificity 67%, PPV 35%, and NPV 90%. the comparable (median) paper-based CANRISK cut-offs are 21, 29, 33 and 43 respectively. These correspond to FINDRISC scores of 7, 12, 15 and 21 respectively. For each FINDRISC category, Figure 4 shows the corresponding mean and 95% confidence interval for pCANRISK scores within the entire FINDRISC category (i.e. not the cut-off score itself). As expected, the CANRISK scores increase monotonically across the FINDRISC categories. This is useful for relating information about future diabetes incidence from the Finnish Diabetes Prevention Study5 to the CANRISK scores. According to FINDRISC,31 more than 1 in 3 high-risk cases would likely develop CANRISK score selected threshold scores include three pCANRISK scores corresponding to FINDRISC cut-off scores in use in Finland, as well as a balanced score. This “optimal score”30 attempts to balance the sensitivity and specificity of the test where the point on the ROC curve is closest to the (0, 1)-point denoting perfect discrimination. It assumes that false positives are equally important as false negatives. The balanced cut-off for pCANRISK is 32. FINDRISC score – 10-year risk categories 27 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada not part of the original FINDRISC scoring metric. As anticipated, ethnicity was strongly predictive of dysglycemia. The OR associated with Aboriginal ethnicity was lower than for some other non-White ethnic groups, as some of this group’s excess risk has been partially captured in other predictors such as BMI, waist circumference and educational attainment. Regarding predictive validity, the AUC for eCANRISK and pCANRISK were both 0.75, indicating that both electronic and paperbased CANRISK scores provide good discrimination30 (i.e. an ability to distinguish true-positive and negative cases based on reference standard blood test results). This means that the predictive validity of both CANRISK scores is confirmed in this multi-ethnic study population. In other words, the AUC results indicate that these prognostic models can effectively distinguish low-risk from high-risk cases. An AUC of 1 would indicate perfect discrimination (100% accuracy), and an AUC of 0.5 would indicate discrimination no better than chance. (A recent review of prognostic models for predicting mortality32 found a median AUC of 0.77 among a total of 94 eligible studies.) The Brier score24 for the model was 0.002, which also indicated good predictive accuracy. These results also indicate that CANRISK is more accurate than the FINDRISC Score model and the simple Obesity model for detecting dysglycemia in this multi-ethnic Canadian population. However, a statistically validated model need not be clinically valid,23 and more research is necessary to establish the clinical utility of the model. Screening thresholds The aim of CANRISK was to develop a simple risk calculator that could be used both in the primary care setting and by individuals themselves. It is first necessary to select CANRISK scores as thresholds. The choice of threshold score will determine the accuracy of CANRISK at that particular cut-off. A lower cut-off score would tend to increase sensitivity but would also increase the number of false positives being referred for follow-up diagnostic testing. The choice of cut-point will also depend on the amount of available resources for subsequent diagnostic testing. The choice of specific cut-off has both potential clinical and economic impli cations; in a clinical setting, the choice would affect the triaged portion referred for follow-up (i.e. diagnostic testing or lifestyle education). For instance, with a paper CANRISK score of 29 as a moderate cut-off, only 50% of CANRISK-assessed cases (i.e. scores 29+) would be referred for follow-up. The remaining 50% of screened-out cases might still receive diagnostic testing on an individual basis at a later date if their family doctor were to order further testing based on symptoms or other clinical indications. Note that these screened-out percentages would likely differ for the eventual target population because the age and ethnic distributions of the overall population would likely differ from those of the core CANRISK sample. For the purpose of validation, the outcome for the prognostic model was based on the collective results of participants’ blood tests (FPG and 2-hour 75g OGTT value). Dysglycemia detection rates based on the FPG alone would have significantly underestimated prevalent dysglycemia: 59% of people with prediabetes and 52% of those with diabetes would have been missed without the 2-hour glucose challenge component of the OGTT. The CANRISK prognostic model therefore presumes that those referred by the risk assessment will receive a diagnostic assessment involving the OGTT. However, a recent Ontario study33 noted that the reference standard OGTT test is under utilized in practice, being used in less than 1% of all diabetes screening tests among asymptomatic adults. This same study33 also found that a significant amount of opportunistic screening effort is already being expended each year to detect diabetes among asymptomatic Canadian adults. Over 63% of adults without diabetes had received a diabetes screening blood test within the previous 3 years. The large majority of this ad hoc screening involves FPG and increasingly HbA1c. An organized Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 28 triaged approach to screening involving CANRISK for initial risk assessment may help increase the cost-effectiveness of detection efforts. We intend to confirm current CANRISK scores by following up the CANRISK cohort in order to assess cumulative diabetes incidence among various ethnic groups and risk categories. For now, the specific variables underlying the current dysglycemia-based CANRISK score aim to broaden the risk assessment discussion with screened participants by quantifying the risks posed by ethnicity, obesity, sex, family history of diabetes, macrosomia and other socio-economic factors. Study limitations Item-missing data was an issue for several variables, particularly for family history of diabetes. In the CANRISK model, it has been assumed that persons who either did not know or who provided no response for history of diabetes for their mother or a sibling were equivalent to “no.” This assumption requires further confirmation through additional data collection and analysis. Other potential sources of response bias may exist due to the self-reported nature of predictor variables. A further limitation of the study was that individual study centres used different eligibility criteria regarding those with previously diagnosed prediabetes (all centres excluded those known to have diabetes). Similarly, during the second phase of their recruitment, one study site (PEI) excluded any persons with prediabetes who were being prescribed the drug metformin. (Most Canadian family physicians do not prescribe metformin for patients with prediabetes but use lifestyle treatment instead.34) Participants in this CANRISK study were recruited as volunteers, not as part of a randomly selected population-based sample. The resulting convenience sample of CANRISK participants does not reflect the proportions of the Canadian population at large. However, obtaining a representative sample was not the primary objective of the study. Rather, the study group was recruited in order to provide sufficient numbers from various major ethnic groups so as to provide adequate statistical power for analyzing ethnicity as a risk factor. As such, the convenience sample developed for this study represents the intended target groups. However, the fact that the sample is not representative of the Canadian population means that the overall perfor mance of the model and the importance of ethnicity (and perhaps some other risk factors) in the general Canadian population may have been over-estimated. Future research Further work would be necessary to determine the acceptability of CANRISK in a clinical setting. For CANRISK to be applied in a clinical context, practical clinical decision rules based on specific cut-off scores will need to be determined by evaluating prospective economic trade-offs between likely resulting costs and health benefits. These decision rules would need to strike a balance between clinical priorities towards maximizing prevention and other practical operational constraints (e.g. testing capacity of local laboratories) concerning the cost of various diabetes screening scenarios. The actual cost of diabetes risk assessment with CANRISK will depend on local circumstances affecting economies of scale in implementation (i.e. scoring thresholds for specific follow-up and testing) and the mode of delivery. A further consideration needs to be the non-monetary costs of false positives (worry) and false negatives (false reassurance). One potential use of CANRISK is in a nonclinical setting by individuals. The utility of CANRISK as an educational tool in this context needs to be investigated. Further research is also required to evaluate practical implementation issues in various settings. The model could be extended to address other specific ethnic groups, such as Latin Americans (i.e. non-White Hispanics), which would help to broaden the applicability of CANRISK to other North American jurisdictions. Current variables describing diet and physical activity could also be enhanced through further data collection and validation studies. The transporta bility of the CANRISK score to other geographic areas and to the Canadian population as a whole will help to further establish the external validity of this new prognostic model. Successful implementation of the CANRISK scoring tool will depend not only on the successful uptake of the risk-scoring questionnaire itself but also on the creation of lifestyle intervention programs for those persons assessed at moderate or high risk of dysglycemia. Current evidence suggests that effective lifestyle change requires a “critical dose of prevention” involving 5 or 6 hours of facilitated discussion over the course of 8 to 12 months.5,6 Based on current economic studies, diabetes prevention stra tegies involving group lifestyle interventions targeted to persons with prediabetes are cost-effective35-37 and may even generate long-term cost savings for the health care system. Conclusion This study has demonstrated that CANRISK is a statistically valid tool that may prove to be suitable for assessing diabetes risk in Canada’s multi-ethnic population. The addition of ethnicity to the basic FINDRISC 29 scoring model improves the ability to distinguish diabetes and prediabetes for early detection and intervention in a Canadian context. Because this new risk assessment tool is both inexpensive and evidence-based, CANRISK may help to enhance the efficiency and effectiveness of targeted diabetes prevention among those at moderate or high risk of developing type 2 diabetes. Acknowledgements The authors wish to acknowledge the contributions of several organizations without whose assistance this research would have been impossible. These contributions reflect data-sharing agreements and research ethics approvals with the following provincial organizations: Health PEI (Charlottetown, Summerside and O’Leary sites), the Diabetes Care Program of Nova Scotia (Kentville and Antigonish sites), New Brunswick Health and Wellness (Fredericton and Lameque sites), Ontario Health and Wellness (the Mississauga site at Credit Valley Hospital), Manitoba Health and Wellness (Brandon and Winnipeg sites), the Saskatoon Regional Health Authority, and the Vancouver Coastal Health Authority. We also wish to acknowledge the valuable comments provided by researchers Markku Peltonen and Jaakko Tuomilehto from Finland’s national public health agency (THL) who reviewed our preliminary results. We also wish to thank the editors at Chronic Diseases and Injuries in Canada who provided useful comments in revising this article. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Appendix Appendix A: Estimating the probability of current dysglycemia based on canrisk scores The probability of current dysglycemia can be estimated for an individual based on either of the following two formulae, depending on whether the score is based on eCANRISK or pCANRISK: (1)Using electronic scores (eCANRISK): 1 Px = 1 + e −(z) where z = α0 + β1 X1 + β2 X2 ...+ βn Xn , such that α0 = −3.842 for the intercept term for the logistic regression model, and βi are the beta coefficients (eCANRISK scores) for each of the respective Xi predictors, from i = 1 to n. Based on the characteristics of the individual mentioned in the main text of the paper (a 58-year-old White man with no other risk factors other than his mother having diabetes), z = −1.929, yielding an absolute risk of 0.13. (2)Using paper-based scores (pCANRISK): 1 Px = 1 + e −(m) where m = α0 + σ (P1 X1 + P2 X2 ...+ Pn Xn), such that α0 = −3.842 for the intercept term, and Pi are the paper-based scores (pCANRISK) for each of the respective Xi predictors, and σ = 0.09393 (i.e. the rescaling factor for converting betas into paper scores). In our example, m = −1.869, yielding an absolute probability of 0.13. References 1. Hypertension in Diabetes Study (HDS): I. Prevalence of hypertension in newly presenting type 2 diabetic patients and the association with risk factors for cardiovascular and diabetic complications. J Hypertens. 1993;11(3):309-17. 2. Harris MI, Klein R, Welborn TA, Knuiman MW. Onset of NIDDM occurs at least 4-7 years before clinical diagnosis. Diabetes Care. 1992; 15(7):815-9. 3. Unwin N, Shaw J, Zimmet P, Alberti KG. Impaired glucose tolerance and impaired fasting glycaemia: the current status on definition and intervention. Diabet Med. 2002;19:708-23. 4. de Vegt F, Dekker JM, Jager A, Hienkens E, Kostense PJ, Stehouwer CD, et al; Finnish Diabetes Prevention Study Group. Relation of impaired fasting and postload glucose with incident type 2 diabetes in a Dutch population: the Hoorn Study. JAMA. 2001;285(16):2109-13. 5. Tuomilehto J, Lindstrom J, Eriksson JG, Valle TT, Hamalainen H, Ilanne-Parikka P, et al. Prevention of type 2 diabetes mellitus by changes in lifestyle among subjects with impaired glucose tolerance. N Engl J Med. 2001;344:1343-50. 6. Knowler WC, Barrett-Connor E, Fowler SE, Haman RF, Lachin JM, Walker EA, et al; The Diabetes Prevention Program Research Group. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med. 2002;346:393-403. 7. Buijsse B, Simmons RK, Griffin SJ, Schulze MB. Risk assessment tools for identifying individuals at risk of developing type 2 diabetes. Epidemiol Rev. 2011;33:46-62. 8. Griffin SJ, Little PS, Hales CN, Kinmonth AL, Wareham NJ. Diabetes risk score: towards earlier detection of type 2 diabetes in general practice. Diabetes Metab Res Rev. 2000 May-Jun;16(3):164-71. 9. He G, Sentell T, Schillinger D. A new public health tool for risk assessment of abnormal glucose levels. Prev Chronic Dis. 2010;7(2):1-9. 10. Heikes KE, Eddy DM, Arondekar B, Schlessinger L. Diabetes Risk Calculator: a simple tool for detecting undiagnosed diabetes and pre-diabetes. Diabetes Care. 2008;31:1040-5. 11. Koopman RJ, Mainous AG, Everett CJ, Carter RE. Tool to assess likelihood of fasting glucose impairment (TAG-IT). Ann Fam Med. 2008;6(6):555-61. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 30 12. Nelson KM, Boyko EJ; Third National Health and Nutrition Examination Survey. Predicting impaired glucose tolerance using common clinical information: data from the third National Health and Nutrition Examination Survey. Diabetes Care. 2003;26(7):2058-62. 13. Park PJ, Griffin SJ, Sargeant L, Wareham NJ. The performance of a risk score in predicting undiagnosed hyperglycemia. Diabetes Care. 2002;25:984-8. 14. Schmidt MI, Duncan BB, Vigo A, Pankow J, Ballantyne CM, Couper D, et al. Detection of undiagnosed diabetes and other hyperglycemia states: the Atherosclerosis Risk in Communities Study. Diabetes Care. 2003;26(5):1338-43. 15. Lindstrom J, Tuomilehto J. The Diabetes Risk Score: a practical tool to predict type 2 diabetes risk. Diabetes Care. 2003;26(3):725-31. 16. Saaristo T, Moilanen L, Jokelainen J, Korpi-Hyovalti E, Vanhala M, Saltevo J, et al. Cardiometabolic profile of people screened for being at high risk of type 2 diabetes in a national diabetes prevention program (FIN-D2D). Prim Care Diabetes. 2010;4:231-9. 17. Saaristo T, Moilanen L, Korpi-Hyovalti E, Vanhala M, Saltevo J, Niskanen L, et al. Lifestyle intervention for prevention of type 2 diabetes in primary health care: one-year follow-up of the Finnish national diabetes prevention program (FIN-D2D). Diab Care 2010;33(10):2146-51. 27. Cowie CC, Harris MI, Silverman RE, Johnson EW, Rust KF. Effect of multiple risk factors on differences between blacks and whites in the prevalence of non-insulin-dependent diabetes mellitus in the United States. Am J Epidemiol. 1993;137:719-32. 18. Kaczorowski J, Robinson C, Nerenberg K. Development of the CANRISK questionnaire to screen for prediabetes and undiagnosed type 2 diabetes. Can J Diabetes. 2009; 33(4):381-5. 28. Brancati FL, Kao WH, Folsom AR, Watson RL, Szklo M. Incident type 2 diabetes mellitus in African American and white adults: the Atherosclerosis Risk in Communities Study. JAMA. 2000;283:2253-9. 19. Papineau D, Fong M. Piloting the CANRISK tool in Vancouver Coastal Health. Chronic Dis Inj Can. 2011;32(1):12-18. 29. Sullivan LM, Massaro JM, D’Agostino RB Sr. Presentation of multivariate data for clinical use: The Framingham Study risk score functions. Stat Med. 2004;23:1631-60. 20. Talbot P, Dunbar M. Nova Scotia Prediabetes Project: upstream screening and community intervention for prediabetes and undiagnosed type 2 diabetes. Chronic Dis Inj Can. 2011;32(1):2-11. 21. World Health Organization; International Diabetes Federation. Definition and diagnosis of diabetes mellitus and intermediate hyperglycemia: report of a WHO/IDF consultation. Geneva (CH): World Health Organization; 2006. 22. Alberti K, Zimmet P; for a WHO Consultation. Definition, diagnosis and classification of diabetes mellitus and its complications. Part 1: Diagnosis and classification of diabetes mellitus. Geneva (CH): World Health Organization; 1999. 23. Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19:453-73. 24. Arkes HR, Dawson NV, Speroff T, Harrell FE Jr, Alzola C, Phillips R, et al. The covariance decomposition of the probability score and its use in evaluating prognostic estimates. Med Decis Making. 1995;15:120-31. 25. SPSS version 15.0 for Windows [statistical software]. Armonk (NY): SPSS Statistics; 2006. 26. Dreyer G, Hull S, Aitken Z, Chesser A, Yaqoob MM. The effect of ethnicity on the prevalence of diabetes and associated chronic kidney disease. QJM. 2009;102:261-9. 37. Hoerger TJ, Hicks KA, Sorensen SW, Herman WH, Ratner RE, Ackermann RT, et al. Cost-effectiveness of screening for pre-diabetes among overweight and obese U.S. adults. Diabetes Care. 2007;30:2874-9. 30. Akobeng AK. Understanding diagnostic tests 3: receiver operating characteristics curves. Acta Pediatr. 2007;96:644-7. 31. Type 2 diabetes risk assessment form [Internet]. [place unknown]: Finnish Diabetes Association; [cited 2011 Sep 9]. Available from: http://www.diabetes.fi /files/1100/Type2diabetesRiskTest_.jpg 32. Siontis GC, Tzoulaki I, Ioannidis JP. Predicting death: an empirical evaluation of predictive tools for mortality. Arch Int Med. 2011 July 25. doi:10.1001 /archinternmed.2011.334. 33. Wilson SE, Lipscombe LL, Rosella LC, Manuel DG. Trends in laboratory testing for diabetes in Ontario, Canada 1995-2005: a population-based study. BMC Health Serv Res. 2009;9:41. 34. Lily M, Godwin M. Treating prediabetes with metformin: systematic review and metaanalysis. Can Fam Physician. 2009;55:363-9. 35. Zhang P, Engelgau M, Norris SL, Gregg EW, Narayan KM. Application of economic analysis to diabetes and diabetes care. Ann Intern Med. 2004;140:972-7. 36. Williamson DF, Vinicor F, Bowman BA; Centers for Disease Control and Prevention Primary Prevention Working Group. Primary prevention of type 2 diabetes mellitus by lifestyle intervention: implications for health policy. Ann Intern Med. 2004;140:951-7. 31 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 2008 Niday Perinatal Database quality audit: report of a quality assurance project S. Dunn, PhD (1,2); J. Bottomley, MHA (3); A. Ali, MSc (4); M. Walker, MD (1,5,6,7) This article has been peer reviewed. Abstract Introduction: This quality assurance project was designed to determine the reliability, completeness and comprehensiveness of the data entered into Niday Perinatal Database. Methods: Quality of the data was measured by comparing data re-abstracted from the patient record to the original data entered into the Niday Perinatal Database. A representative sample of hospitals in Ontario was selected and a random sample of 100 linked mother and newborn charts were audited for each site. A subset of 33 variables (representing 96 data fields) from the Niday dataset was chosen for re-abstraction. Results: Of the data fields for which Cohen’s kappa statistic or intraclass correlation coefficient (ICC) was calculated, 44% showed substantial or almost perfect agreement (beyond chance). However, about 17% showed less than 95% agreement and a kappa or ICC value of less than 60% indicating only slight, fair or moderate agreement (beyond chance). Discussion: Recommendations to improve the quality of these data fields are presented. Keywords: audit, data quality, quality assurance, reliability Background The Ministry of Health and Long-Term Care (MOHLTC) in Ontario recognized that producing and sustaining quality surveillance data is the foundation of an effective and efficient health system.1 Surveillance is defined as the ongoing systematic collection, analysis and interpretation of health data essential to the planning, implementation and evaluation of public health practices, integrated with the timely dissemination of these data to key stakeholders.2 A surveillance system can function as both measurement tool and stimulus for action3 by providing early warning of health problems and evidence for policy and program development, risk assessment, trend analysis and the evaluation of prevention and control strategies.4 However, the usefulness of a surveillance system is limited by the quality of the data it collects and analyzes. In Ontario, the Niday Perinatal Database (the “Niday”) is the source of data to assess outcomes, risk factors and interventions related to perinatal care. It was created in 1997 under the direction of the Perinatal Partnership Program of Eastern and Southeastern Ontario (PPPESO) to provide perinatal data to PPPESO partners. This Internet-based system has evolved significantly since its inception and has become a unique co-operative venture with over 100 health care organizations across the province contributing real-time perinatal data. It enhances the ability of health care providers in different parts of the province and within different service sectors to work together to improve perinatal health. At the time of the audit, 96% of Ontario births were captured in the Niday, and there were 90 defined patient elements covering the full spectrum of perinatal care (Table 1). In 2001, the province adopted the variables in the Niday as the minimum dataset. This is the only database in Ontario that provides immediate access to real-time population-based perinatal data for an entire region. The Better Outcomes Registry and Network (BORN Ontario) Steering Committee now manages the project. The involvement of most hospitals in the province also permits inter-hospital/health unit comparisons necessary for benchmarking and performance improvement based on learning from others’ successes. As the system evolves, BORN is committed to ensuring high quality data, with powerful and efficient reporting tools.5 In light of the fact that approximately 40% of all live births in Canada occur in Ontario (37.1% in 2008/2009),6 this database provides rich perinatal information for a large proportion of the births in Canada. Although it is well recognized that the foundation of an effective and efficient health system requires the production of quality data,1 it was unclear whether the Niday, as configured, was a reliable source of information. The goal of this quality assurance project was to assess objectively the reliability, completeness and comprehensiveness of the data in the Niday Perinatal Database. Author references: 1. Better Outcomes Registry and Network (BORN Ontario), Ottawa, Ontario, Canada 2. Champlain Maternal Newborn Regional Program (CMNRP), Ottawa, Ontario, Canada 3. Children’s Hospital of Eastern Ontario, Ottawa, Ontario, Canada 4. Ottawa Public Health, Ottawa, Ontario, Canada 5. OMNI Research Group, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada 6. Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, Faculty of Medicine, Ottawa General Hospital, University of Ottawa, Ottawa, Ontario, Canada 7. Tier 1 Canada Research Chair, Perinatal Epidemiology, University of Ottawa, Ottawa, Ontario, Canada Correspondence: Sandra Dunn, BORN Ontario, The Ottawa Hospital, 501 Smyth Rd. Room 1818, Box 241, Ottawa ON K1H 8L6; Tel.: (613) 737-8899 ext. 72070; Email: [email protected] Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 32 Table 1 List of variables in the Niday Perinatal Database in 2008 (n = 90) including variables chosen for re-abstraction as part of the 2008 quality audit City/towna Mother’s agea Siteb Maternal chart numberb Mother’s birth date Postal code Languagec Aboriginalc Previous Caesarian section Number of previous Caesarian sectionsd Maternal health problemsc Obstetrical complicationsc GBS screeningd GBS (35–37 weeks) resultsd Maternal transfer from Labour type If induced, indication (17) If induced, method (8) Number of induction attemptsc Augmentationd Intrapartum complicationsc Maternal pain management (11) Fetal surveillance (6) GBS antibioticsd Antenatal steroids Labour/birth commentd Forceps/vacuum Newborn resuscitation (7) Baby’s sex Gestational age Birth weight Apgar score 1 Apgar score 5 Apgar score 10c Infant feeding in hospitalc Reason for breastmilk substitutec Infant feeding on dischargec Hearing screeningc HBHC screenc HBHC screen if not sent, why?c Arterial cord pHc Birth nurse ID Birth physician ID Discharge time Mother’s date of admission Mother’s time of admission Mother’s height (centimetre) Linked data Provincea Record typea Identifying variables Baby chart numberb Baby birth dateb Maternal history variables If transferred, reasonc Antenatal care providerc First trimester visitc Prenatal classesc Smoking Intention to breastfeedd Number of previous term babies Number of previous preterm babies Reproductive assistancec Multiple gestation Maternal history commentd Labour and birth variables Episiotomy Laceration Presentationd Delivery type If Caesarian section, indication (20) If Caesarian section, typed If Caesarian section, dilatationc Time fully dilatedc Time start pushingc Time of birth Delivered by Newborn variables Arterial base excessc Venous cord pHc Venous base excessc Congenital anomaliesc Phototherapyc Newborn commentd Neonatal death / stillbirth Neonatal discharge / transfer datec Neonatal discharge / transfer timed Discharge weightc Discharged / transferred toc Reason for neonatal transferc Neonatal transfer hospital User-defined variables fieldse Removal of placenta Mother’s weight (kilogram) Newborn drug screening Newborn drug screen results Abbreviations: GBS, Group B Streptococcus; HBHC, Healthy Babies Healthy Children. Notes: Total variables in Niday Perinatal Database in 2008 (n = 90): Mandatory 24 + Non-mandatory 66. Total number of variables included in re-abstraction (n = 33/90; 36.7% - resulting in 96 data fields for audit). Mandatory variables (n = 20/90) (4 providedb). Non-mandatory variables (n = 13/90). a Mandatory variables – linked data (n = 4/90; 4.4%). b Provided identifying labels. c Missing > 10% data (n = 31/90; 34.4%). d Not identified as a priority at the time of the audit (n = 12/90; 13.3%). e User defined variables (n = 10/90; 11.1%) – not available to all sites. 33 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Methods The Data Quality Management Framework,7 developed by the MOHLTC Health Results Team for Information Management, was used to guide this project. According to the Tri-council policy, and given the fact this was a quality assurance project, Research Ethics Board approval was not required.8 Hospital participation in this project was voluntary, and every effort was made to ensure the confidentiality of patient information and privacy of participating hospitals. Data re-abstraction In order to determine the reliability and completeness of the data, re-abstraction of information from patient records was carried out to assess agreement between selected variables in the perinatal database and the mother and infant charts. Written consent was requested from and given by each site participating in the re-abstraction phase of the project. Information was handled confidentially, and each auditor signed a Pledge of Confidentiality Form. The auditors re-entered data from the patient records that had already been collected and entered by the hospital data entry person into the Niday. The laptops used for data entry were supplied to the auditors and returned following the re-abstraction process. The electronic data were then securely transferred to the statistician for analysis and deleted from the laptops. Data were aggregated for analysis, and findings were anonymized. Setting and sample size (hospitals) Purposive sampling was used to recruit 14 hospitals for the audit representing five regions of the province: East/Southeast, Greater Toronto Area (GTA), Central West, South West, and North. The sample captured both obstetrical and newborn care practices and included all levels of care: level 1, or low-risk pregnancies (4 of 51 hospitals in Ontario); level 2, or women/ babies with health problems (8 of 37 hospitals in Ontario); and level 3, or specialized care (2 of 7 hospitals in Ontario). A combination of both paper and electronic documentation systems and a variety of data entry processes were used by the sample hospitals. Sample size A computer-generated random sample of 100 maternal chart numbers (and linked baby records) was identified for each participating site from existing records that had already been entered into the Niday in 2008 (total of 200 charts per site). The total sample size for this project was 1395 linked mother-baby dyads; in three cases the patient charts could not be located at the time of the re-abstraction, and in two cases the chart numbers were not for a perinatal client. Each of the auditors was told about the project and trained in the re-abstraction process, including where to find the information in the patient record and how to use the SPSS (version 15.0) spreadsheet for data collection to ensure consistent re-abstraction. Each received a handout containing the definition of terms for each of the variables in the Niday, contact information for the project coordinator, a list of their designated hospital(s) and an SPSS spreadsheet with pre-entered sample data (maternal chart number, baby chart number, baby date of birth) for each of their designated sites. For practice, the auditors entered data into the SPSS spreadsheet based on the same two charts; inter-rater reliability was evaluated based on these cases. Variables for re-abstraction Data collection procedure A subset of variables (33/90; 36.7%) from the Niday perinatal dataset was chosen for re-abstraction. Selection was based on the following criteria: a) a mandatory variable; b) a non-mandatory variable with less than 10% missing data based on verification reports; and c) a variable that addressed a practice issue of interest (e.g. use of antenatal steroids, indication for Caesarean section, episiotomy, lacerations, fetal surveillance, forceps/vacuum, indications for induction, method of induction, maternal pain relief, smoking). This resulted in 96 data fields available for re-abstraction because some of the variables consisted of multiple data fields (e.g. indications for induction included 17 data fields; maternal pain mana gement included 11 data fields). Table 1 lists the variables selected for re-abstraction and those excluded (with rationale). Auditors Due to the wide geographic distribution of the participating hospitals, and the travel and time involved to complete an audit in 14 sites across the province, six auditors with a health care background were hired and trained to expedite the process. Two auditors entered data at five sites each and each of the remaining four auditors re-abstracted data at one site each. Figure 1 shows a flow sheet of the data collection process. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 34 Following the random chart selection process, a list of the patient records from each of the participating hospitals was prepared. The identifying variables used were the mother’s chart number and the matching baby’s chart number. For added precision, the date of birth was printed out for each baby. This enabled auditors to verify that the record entered was the correct one. In each of the 14 participating hospitals, a key contact person was identified and informed about the project by the project manager. The key contact was asked to assist (or designate someone who could assist) the auditors to obtain entry to the site, access the patient charts from health records and problem-solve any site-related issues. Prior to data collection, the key contact person (or designate) met with the auditor to show the patient documentation systems and where to find the key information. Primary data abstraction took place from April to July 2008. Data collection for one site had to be repeated in October 2008 as the original file for this site was overwritten and the data were lost. The charts (paper or electronic records) were obtained from the Health Records Departments of each of the participating hospitals. The auditors reviewed and Perinatal Database. Cross-tabulations were generated to explore non-agreements and missing data in an attempt to identify potential reasons for the variation between the auditor and the original data entered for each field. Figure 1 The data collection process 2008 Niday Database File Random selection process: 100 cases (matched mother/baby charts) from each of 14 hospitals Data collector 1 • 5 sites (A, B, C, D and E) • 498 cases • 996 patient charts reviewed Data collector 4 • 1 site (H) • 100 cases • 200 patient charts reviewed Data collector 2 • 1 site (F) • 100 cases • 200 patient charts reviewed Data collector 5 • 1 site (I) • 100 cases • 200 patient charts reviewed Data collector 3 • 1 site (G) • 100 cases • 200 patient charts reviewed Data collector 6 • 5 sites (J, K, L, M and N) • 497 cases • 994 patient charts reviewed Database Audit File Total of 1395 records re-abstracted re-abstracted the data using the standardized data entry procedures. The data were collected using an SPSS version 15 data file template. A spreadsheet was created that included the data fields under review and pull-down menus matching those found on the current Niday entry screen. For ease of data entry, the variables were placed in the same order as they appeared in the majority of hospital records. Data were entered into two portable laptop computers. Re-abstraction took two to four days per site, due to standard delays when accessing patient records and the time it takes to work through the information in each patient record. The project manager was available by pager, phone or email during the re-abstraction process to address any questions that arose. code, mother’s age and maternal transfer from another hospital were obtained from the admission record; the rest of the variables were obtained from the labour record, the delivery record, the antenatal record, the discharge summary, lab results, nurses’ notes, doctors’ orders, medication records and the postpartum screening record. Terminology and the organization of the patient chart varied somewhat from site to site, but the overall layout of the information was similar. In one region, a standardized documentation system was used by all of the parti cipating hospitals except one. All of the records were in either English or English/French. Although sensitivity and specificity can be used to measure the accuracy of data gathered from an external source compared to a primary source of infor mation, this approach requires that one of the data sources is identified as the gold standard.9 Many factors can affect the transfer of information from a patient record, such as observer variation, poor documentation, illegible charts, data loss, unavailability and timeliness of chart completion.10 This makes it impossible to identify a gold standard from either the original data entered into Niday or the re-abstracted data entered by the auditors. When neither data source can be designated as the gold standard, high agreement between the two suggests high reliability. In other words, when two similar datasets are compared and a high proportion of the data are the same, then it can most likely be interpreted that they are both correct. This is an indicator of having high quality data. Therefore, for the purposes of this audit, we used percent agreement, Cohen’s kappa statistic and intraclass correlation coefficient (ICC) between the variables11 to compare the data newly re-abstracted from patient records with data previously entered into the Niday by the participating hospitals. Percent agreement was calculated for all variables. For kappa and ICC, categorical/ nominal variables (n = 87), and continuous variables (n = 3) were considered separately. Categorical variables Analysis Patient records Although hospital patient documentation systems are not standardized throughout the province, the chart reviews were conducted as consistently as possible. Auditors were trained to obtain information from the same sources used for the original data entry. The postal Descriptive statistics (frequencies, means and percentages) were calculated using SPSS version 15 to describe the characteristics of the study sample groups. The reliability of the data was assessed by comparing the re-abstracted data from the patient record to the original data entered into the Niday 35 The analysis for all the categorical/ nominal variables (except for postal code) was by two-way cross tabulations of each variable and comparison of the entries, as explained above. Since postal codes are string variables, cross tabulation was not feasible so an equivalent equal/not equal statement on the SPSS program was used to calculate the percent agreement. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada We used Cohen’s kappa statistic to examine the proportion of responses in agreement in relation to the proportion of responses that would be expected by chance, given symmetrical marginal distributions.12-14 Cohen’s kappa statistic represents the proportion of agreements after chance agreement has been excluded. Kappa values range from 0 (no agreement) to 1 (total agreement). According to Landis and Koch, a kappa value of 0.90 (or 90%) indicates almost perfect agreement while a kappa value of 0.55 (or 55%) reflects only moderate agreement.15 Continuous variables For continuous variables, agreement was assessed using an equal/not equal statement on the SPSS program and by calculating the ICC. ICC is a more appropriate measure of reliability for continuous data than Pearson’s product moment correlation coefficient or Spearman’s rank-order correlation coef ficient since these measure association rather than agreement.12-14 ICC values range between 0 (no agreement) and 1 (total agreement), “with values approaching 1 representing good reliability.”16, pg. 357 According to Portney and Watkins,17 an ICC of over 0.9 (or 90%) indicates excellent agreement, while an ICC of 0.35 (or 35%) indicates poor agreement between variables. The notes to Table 2 shows more detailed interpretation of kappa and ICC values. Results This quality assurance project evaluated the reliability, completeness and comprehensiveness of the Niday Perinatal Database and found that the database met expectations either fully or partially. Reliability A total of 33 out of 90 variables (96 data fields) in the Niday were re-abstracted from patient records to determine the degree of agreement with data already entered in the database. Of the 89 data fields for which kappa or ICC was calculated, almost one-half (n = 39; 43.8%) showed substantial or almost perfect agreement (beyond chance), suggesting that these variables may be used with confidence. Just over one-third of the data fields (n = 34; 38.2%) were found to have kappa values below the moderate level (60% beyond chance) despite having excellent agreement rates. However, a prevalence effect due to asymmetrical imbalances of marginal totals was the likely cause of the low kappa value in this group.18 The remaining data fields (n = 15; 16.9%) showed both percent agreement of less than 95% and a kappa or ICC value less than 60% indicating only slight, fair, poor or moderate agreement (beyond chance). This suggests these data fields may be problematic and require further inves tigation. Table 2 summarizes the percent agreements, Cohen’s kappa or ICC for each data field. Completeness Approximately 34% of the variables in the Niday were missing more than 10% of data based on verification reports generated prior to the start of the audit. Only variables that were mandatory or had low rates of missing data (< 10%) just prior to the audit were selected for re-abstraction (Table 1). Missing (not entered) data were also evaluated as part of the re-abstraction and were found to be associated with the following variables: antenatal steroids, forceps/vacuum, episiotomy, laceration and smoking. The missing data were limited to only three sites (F, J and K; see Figure 1). The primary reason for missing data at these sites was due the auditors or original hospital data entry personnel deciding to leave a cell empty rather than selecting “none” or “unknown.” At site F the auditor left the field empty while the hospital data entry person entered “none” or “unknown,” while the reverse took place at sites J and K. Missing data was not a significant issue and these data points were not excluded from the assessment of agreement. This was not a surprising finding, given the fact that these variables were selected for abstraction in the first place because of high completion rates. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 36 Comprehensiveness At the time of the audit over 96% of births in the province (involving 95 delivering hospitals and including midwifery hospital births and some home births) were captured in the Niday. There were 90 defined patient elements with 23 mandatory fields (at the start of the audit). Discussion Although neither of the datasets used during the audit can be declared as a gold standard, the moderate-to-high levels of agreement (beyond chance) between the two sources suggest that the variables are comparable across two methods of data collection.19 The worst case scenario in interpreting these findings would be that all the differences are due to having wrong data in the Niday. When there is a level of disagreement between the two data sources for some data fields, part of this difference may be as a result of wrong data in the Niday, wrong data entered during the audit, or wrong data in both datasets. Although the reasons for non-agreements could not always be discerned, a variety of potential factors were identified during detailed exploration of the data. Results from the audit indicated disagreement between the two data sources occurred across multiple sites, and included both hospital and auditor data entry issues. These issues have been clustered into four themes (data entry choice, clarity of information, inaccurate documentation and human error). The first issue related to choices available for data entry has to do with the designation given to some variables. At the time of the audit, all data fields in the Niday were designated as either mandatory or non-mandatory. In reviewing non-agreements, it was evident that in some cases the auditor found information in the patient record that the original hospital data entry person did not record. Although, both groups were tasked with finding and entering as much information as possible, in reality it is possible that discretionary completion of some of the non-mandatory Table 2 Comparison of abstracted data from patient records (N = 1395) and data entered in Niday Perinatal Database using percent agreement, Cohen’s kappa and intraclass correlation coefficient (ICC) No. Variable Name Data Field Label Coding Not matched n/1395 (%) Percent agreement (%) Cohen’s kappa [ ] (%) ICC (%) 1. SITE Site name Pre-entered 2. Maternal chart number Maternal chart no. Pre-entered 3. Baby chart number Baby chart no. Pre-entered 4. Baby birth date Baby birth date – DMY Pre-entered 5. Number of previous preterm babies No previous preterm babies Number (0–15) Unknown 64 (4.6) 95.4 54.5 6. Number of previous term babies No previous term babies Number (0–15) Unknown 79 (5.7) 94.3 91.2 7. Previous Caesarean section Previous C/S Yes No Unknown 50 (3.6) 96.4 81.8 8. Maternal transfer from Maternal transfer from Pick from site list Planned home birth Out of region No transfer 35 (2.5) 97.5 25.0 9. Multiple gestation Multiple gestation Singleton Twin Triplet Quadruplet Quintuplet Sextuplet Septuplet 1 (0.1) 99.9 98.8 10. Labour type Labour type Spontaneous Induced No labour 135 (9.7) 90.3 81.8 11. Delivery type Delivery type Vaginal Caesarean section Unknown 4 (0.3) 99.7 97.3 12. Mother’s birth date Mother’s birth date – DMY Date of birth (D/M/Y) 128 (9.2) 90.8 N/Aa 13. Birth weight Birth weightb,c Birth weight (grams) 114 (8.2) 91.8 35.1 14. Gestational age at birth Gestational age at birth Gestational age (weeks) Unknown 119 (8.5) 91.5 32.0 15. Baby’s sex Baby gender Male Female Ambiguous Unknown 29 (2.1) 97.9 96.0 16. APGAR – 1 APGAR1 Number (0–10) Unknown 58 (4.2) 95.8 92.5 17. APGAR – 5 APGAR5 Number (0–10) Unknown 51 (3.7) 96.3 87.7 18. Newborn resuscitation Noneb Not checked Checked 352 (25.2) 74.8 46.7 12 (0.9) 99.1 64.3 Mandatory data fields b 19. Drugs 20. FF02 118 (8.5) 91.5 70.2 21. Intubation 10 (0.7) 99.3 63.9 22. PPV 54 (3.9) 96.1 63.4 23. Chest Compression 5 (0.4) 99.6 28.4 24. Unknown 86 (6.2) 93.8 3.0 11 (0.8) 99.2 50.0 25. b,c Neonatal transfer to Neonatal transfer hospital Pick from site list No transfer (if birth hospital) Out of region N/Aa Continued on the following pages 37 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Table 2 (continued) Comparison of abstracted data from patient records (N = 1395) and data entered in Niday Perinatal Database using percent agreement, Cohen’s kappa and intraclass correlation coefficient (ICC) No. Variable Name Data Field Label Coding 26. Neonatal death / stillbirth Neonatal death / stillbirth Not matched n/1395 (%) Percent agreement (%) Cohen’s kappa [ ] (%) 2 (0.1) 99.9 50.0 97 (7.0) 93.0 N/Aa ICC (%) Mandatory data fields (continued) Not applicable Stillbirth ≥ 20 weeks Neonatal death < 7 days Neonatal death > 7–28 days Non-mandatory data fields 27. Maternal postal code Maternal postal code Full postal code 28. Antenatal steroids Antenatal steroids None 1 dose < 24 hr 2 doses: last dose < 24 hours 2 doses: last dose ≥ 24 hours Unknown 354 (25.4) 74.6 7.5 29. Fetal surveillance FS – Admission stripb,c Not checked Checked 424 (30.4) 69.6 39.2 b,c 30. FS – Auscultationb,c 263 (18.9) 81.1 60.0 31. FS – Intrapartum electronic fetal monitoring (external)b,c 265 (19.0) 81.0 53.2 32. FS – Intrapartum electronic fetal monitoring (internal)b,c 125 (9.0) 91.0 45.0 33. FS – No Monitoring 29 (2.1) 97.9 11.4 34. 35. 36. FS – Unknown If induced – indication for induction None Diabetes Not checked Checked 36 (2.6) 97.4 13.5 10 (0.7) 99.3 12.5 9 (0.6) 99.4 74.0 37. Elective 31 (2.2) 97.8 26.8 38. IUGR/SGA 14 (1.0) 99.0 64.5 39. LGA 40. Maternal obstetrical conditions 8 (0.6) 99.4 55.3 32 (2.3) 97.7 14.6 41. Multiple gestation 4 (0.3) 99.7 66.5 42. Non-reactive NST 5 (0.4) 99.6 28.4 43. Oligohydramnios 7 (0.5) 99.5 79.8 44. Poor biophysical score 5 (0.4) 99.6 28.4 45. Post dates 64 (4.6) 95.4 73.8 46. Pre-eclampsia 25 (1.8) 98.2 43.6 47. Pre-existing maternal medical conditions 6 (0.4) 99.6 24.8 48. PROM 42 (3.0) 97.0 52.8 49. Other maternal 51 (3.7) 96.3 32.1 50. Other fetal 24 (1.7) 98.3 32.5 51. Other 16 (1.1) 98.9 24.5 52. 53. 54. If induced – method of induction None Amniotomyb Not checked Checked Cervidil 2 (0.1) 99.9 85.0 125 (9.0) 91.0 51.2 53 (3.8) 96.2 70.0 55. Cytotec/Misoprostol 15 (1.1) 98.9 20.5 56. Mechanical 10 (0.7) 99.3 63.9 57. Oxytocin 58. Other 129 (9.2) 90.8 66.1 26 (1.9) 98.1 18.0 N/Aa Continued on the following pages Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 38 Table 2 (continued) Comparison of abstracted data from patient records (N = 1395) and data entered in Niday Perinatal Database using percent agreement, Cohen’s kappa and intraclass correlation coefficient (ICC) No. Variable Name Data Field Label Coding Not matched n/1395 (%) Percent agreement (%) Cohen’s kappa [ ] (%) 31 (2.2) 97.8 38.3 ICC (%) Non-mandatory data fields (continued) 59. 60. Other – Prostaglandin If Caesarian section – indication for Caesarian section 2 (0.1) 99.9 85.0 21 (1.5) 98.5 82.4 Cord prolapse 1 (0.1) 99.9 80.0 63. Diabetes 7 (0.5) 99.5 49.0 64. Failed forceps/vacuum 3 (0.2) 99.8 72.6 61. 62. None Breech Not checked Checked 65. Fetal anomaly 0 100.0 100.0 66. IUGR/SGA 5 (0.4) 99.6 54.4 67. LGA 4 (0.3) 99.7 33.3 68. Maternal request 26 (1.9) 98.1 17.9 69. Multiple gestation 12 (0.9) 99.1 64.3 70. Non-progressive labour / descent / dystocia 34 (2.4) 97.6 76.6 71. Non-reassuring fetal status 31 (2.2) 97.8 72.3 72. Placenta previa 1 (0.1) 99.9 90.9 73. Placental abruption 4 (0.3) 99.7 60.0 74. Preeclampsia 8 (0.6) 99.4 42.6 75. Prematurity 8 (0.6) 99.4 19.8 76. Previous Caesarean 22 (1.6) 98.4 89.7 77. PROM 4 (0.3) 99.7 60.0 78. Other fetal health problem 14 (1.0) 99.0 50.0 79. Other maternal health problem 17 (1.2) 98.8 31.4 80. Forceps vacuum Forceps/vacuumb None Forceps Vacuum Forceps and vacuum Unknown 189 (13.5) 86.5 55.5 81. Episiotomy Episiotomyb None Mediolateral Midline 3rd degree extension 4th degree extension Unknown 241 (17.3) 82.7 46.9 82. Laceration Laceration None 1st degree 2nd degree 3rd degree 4th degree Cervical tear Other Unknown 347 (24.9) 75.1 63.0 83. Maternal pain relief None Not checked Checked 69 (4.9) 95.1 52.4 101 (7.2) 92.8 85.5 8 (0.6) 99.4 73.1 111 (8.0) 92.0 45.8 97 (7.0) 93.0 84. Epidural 85. General 86. Local 87. Narcotics b 39 82.4 Continued on the following page Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Table 2 (continued) Comparison of abstracted data from patient records (N = 1395) and data entered in Niday Perinatal Database using percent agreement, Cohen’s kappa and intraclass correlation coefficient (ICC) No. Variable Name Data Field Label Coding Not matched n/1395 (%) Percent agreement (%) Cohen’s kappa [ ] (%) 94 (6.7) 93.3 71.9 319 (22.9) 77.1 49.5 1 (0.1) 99.1 92.3 ICC (%) Non-mandatory data fields (continued) 88. Nitrous Oxide 89. Non-pharmacological 90. Pudendal 91. Spinal epidural combination 21 (1.5) 98.5 50.4 92. Spinal 51 (3.7) 96.3 85.3 93. b Unknown 15 (1.1) 98.9 46.0 127 (9.1) 90.9 N/Aa Obstetrician Family physician Midwife at hospital Midwife at home Nurse practitioner Specified midwife group Other Unknown 159 (11.4) 88.6 71.8 No smoking ≤ 20 weeks > 20 weeks ≤ 20 and > 20 weeks Unknown 294 (21.1) 78.9 50.7 94. Time of birth Time of birth Time of birth (24 hour format) None 95. Delivered by Delivered by 96. Smoking status Smokingb,c N/Aa Abbreviations: FF02, free flow oxygen; FS, fetal surveillance; ICC, intraclass correlation coefficient; IUGR, intrauterine growth restriction; LGA, large for gestational age; NST, non-stress test; PPV, positive pressure ventilation; PROM, premature rupture of membranes; SGA, small for gestational age. Notes: Cohen’s kappa statistic ( ) degrees of agreement after chance agreement has been excluded15: Poor < 0; Slight = 0–0.20; Fair = 0.21–0.40; Moderate = 0.41–0.60; Substantial = 0.61–0.80; Almost perfect = 0.81–1.00. Intraclass correlation coefficient (ICC) degrees of agreement17: Poor < 0.50; Moderate = 0.50–0.75; Good ≥ 0.75–0.90; Excellent > 0.90. a N/A – not applicable as equal/not equal was used, hence no cross tabulation to generate the kappa statistic. b Data fields with < 95% agreement and kappa or ICC values < 60% indicating only slight, fair, poor or moderate agreement (beyond chance). c Data fields also found to be problematic during a previous audit of the Niday Perinatal Database.20 data fields at some sites contributed to the non-agreements. This example illustrates the importance of ensuring that all data fields are mandatory and that only essential, meaningful data are collected. The second issue related to this theme was about pick-list choices and the availability of information in the patient health record. If the information is not documented in the patient record in such a way as to match the pick-list choices, data quality can be affected. For example, in the case of smoking during pregnancy, documentation may indicate that a women smoked, but not provide the detail required to determine the duration of smoking through pregnancy (e.g. above or below 20 weeks as required for Niday at the time of the audit). In some cases where non-agreement occurred, it was because some people entered “unknown” while others left the field empty when the required data was not available in the patient health record. This example illustrates the importance of aligning documentation tools with data entry processes to enhance data quality. The second theme has to do with clarity of information available for each data field. Confusion over the wording, use of double negatives and different interpretations of the definitions for some variables may have contributed to non-agreements (e.g. interpreting what qualifies as an induction or augmentation of labour). This example illustrates the Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 40 importance of ensuring the definitions for each variable are precise and applicable to practice. The third theme was related to inadequate, illegible or inaccurate documentation. Data entry is dependent on the accuracy of the information recorded in the patient health record. Even though specific documents were identified to be the source of information for data entry for both the primary and audited datasets, some of the information entered was difficult to find, or inconsistent, contributing to non-agreement. For example, gestational age and birth weight both require double entry of the data. Double entry of these variables may provide verification that the original number entered is correct, which enhances reliability of the variable, but it does not ensure validity of the information. This is evidenced by the discrepancies between the original data entered and the auditors’ data for these variables. Finally, even though every attempt was made to ensure a consistent process for data entry, it is always possible that human error contributed to non-agreements between the two datasets. Results of this audit have provided information about potential issues related to data entry for some variables in the database. A number of variables were more problematic. Further exploration of the issues is required in order to develop strategies to improve the data quality for these variables in the Niday. Interestingly, eight of the data fields identified in this audit as less reliable were also found to be problematic during a previous audit of the Niday (Table 2).20 This is significant in that some of these variables have been identified as priority items highly relevant for the perinatal reports being developed by BORN Ontario. A previous validation study that explored record linkage of births and infant deaths in Canada examined gestational age and birth weight and indicated good overall agreement.21,22 Gestational age was also found to have a relatively high degree of agreement between the Discharge Abstract Database (DAD) of the Canadian Institute for Health Information (CIHI) and the Nova Scotia Atlee Perinatal Database (NSAPD).23 This finding is in contrast to our study, where gestational age and birth weight achieved ICC values of between 30% and 40%, indicating poor agreement. Ensuring completeness and reliability of the data entered into the Niday is a challenge. Data are entered manually via a secure Internet website or uploaded directly into the database from electronic documentation systems. Regional coordinators send reminders to hospital staff to facilitate the process of data entry and to troubleshoot problems when needed. Verification reports are generated quarterly by a data analyst to identify inconsistencies in numbers and types of births and find errors in the data. A training program has been developed so that all users have a thorough understanding of the system. Sustainability of this database depends on achieving broad support at all levels and valuing the system as a key attribute of the patient safety movement. Based on the results of this audit, and through consultation with experts in the field, a number of recommendations have been put forward to improve data quality (Table 3). This audit is in line with the MOHLTC quality assurance initiatives, and it is a logical step to improving data quality and perinatal care practices. The Niday Perinatal Database is a comprehensive, multifaceted system providing data to perinatal care providers, decision makers, educators and researchers in Ontario. Since the audit, the Niday has expanded to capture data for 100% of births in the province. Many upgrades and improvements to the system have already been completed. Further exploration of quality issues is ongoing as part of the initiative to integrate the database with four other perinatal/ newborn databases (Fetal Alert Network, Maternal Multiple Marker Screening, Newborn Screening, and the Ontario Midwifery Program (OMP) Database. Recent Ministry funding and a newly established administrative body (BORN Ontario) have been established to carry these recommendations forward. Limitations There are two potential limitations to this audit: completeness and clarity of the patient health record and sampling method. Of the hospitals entering data into the Niday at the time of the audit, 14% were recruited to participate in the re-abstraction process. This sample pool was sufficient to identify a number of issues. Although, the patient charts were selected randomly, the hospitals were selected through purposive sampling; therefore, the results of these analyses may not be generalizable to all hospitals in the province. Data entry personnel for both the original data entry to the Niday database and the re-abstraction process were asked to collect as much Table 3 Recommendations to improve quality of data 1. Establish a system of ongoing surveillance of data quality in each organization; 2. Encourage participating hospitals to promptly correct any data entry errors identified through the verification reports; 3. Identify and communicate corrective action to reduce occurrence of recurring errors; 4. Reinforce the need to ensure accurate documentation at point of care and to ensure access to information for data entry personnel; 5. Re-evaluate and monitor use of terms (e.g. none and unknown); Caesarean delivery was found to be coded accurately in the DAD, and information on first to fourth degree perinatal lacerations and induction of labour was also reasonably accurate in this study.23 Results of our audit were consistent with respect to delivery type and lacerations, with substantial or almost perfect agreement (beyond chance) achieved between the re-abstracted data and the information previously entered into the Niday. However, induction method (amniotomy) was less reliable with only 51.2% agreement (beyond chance) noted between the two datasets. 6. Establish automatic verification checks at the time of data entry (i.e. birth weight, gestational age, maternal data of birth, postal code); 7. Build in logic checks (i.e. logic checks based on Neonatal Resuscitation Program standards); 8. Set birth weight limits based on gestational age but allow override capability; 9. Reassess variable options (i.e. antenatal steroids, episiotomy, lacerations, forceps/vacuum, maternal pain relief, newborn resuscitation, smoking status); 10.Clarify definitions for the following variables: delivered by; fetal surveillance (intrapartum fetal monitoring internal or external, admission strip, auscultation); method of induction (amniotomy); labour type (induced); and augmentation; 11.Require mandatory completion of essential variables (i.e. those required for reporting), reinforce use of standard data entry worksheets; 12.Provide ongoing training to ensure that all data entry personnel have had standardized training in data entry; and 13.Use data dictionaries to ensure that everyone understands the options for each variable. 41 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada information as possible from the patient chart and to be vigilant in entering the data. However, reliability of the data entered into the Niday database is dependent on completeness and clarity of the information documented. Deficits in either regard can influence the reliability of the data entered and influence the results of an audit. 2. Last JM. A dictionary of epidemiology. 4th ed. New York (NY): Oxford University Press; 2001. 3. Health Canada. The Canadian Perinatal Health Report. Ottawa (ON): Minister of Public Works and Government Services Canada; 2000. 13. Norman GR, Streiner DL. Biostatistics: The bare essentials. 2nd ed. Hamilton (ON): BC Decker; 2000. Conclusions 4. Choi BC. Perspectives on epidemiologic surveillance in the 21st century. Chronic Dis Can. 1998;19(4):145-51. 14. Bartko JJ. Measurement and reliability: statistical thinking considerations. Schizophr Bull. 1991;17(3):483-9. 5. BORN Ontario [Internet]. Ottawa (ON): BORN [updated 2011] Mission and Vision [cited 2011 Apr 26]. Available from: http://www.bornontario.ca/about-born /vision-and-mission 15. Landis RJ, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-74. There were 90 defined patient elements within the Niday Perinatal Database at the start of the audit. Approximately one-third of the variables were re-abstracted from the patient record to determine agreement with the data already entered in the Niday Database. Approximately 17% of the data fields audited showed both percent agreement of less than 95% and a kappa or ICC value of less than 60%, indicating only slight, fair, poor or moderate agreement (beyond chance) between the data originally entered into the Niday database and the data re-entered during the audit. This suggests these data fields may be less than reliable and require further investigation to ensure quality. Acknowledgements This project is the result of the effort of many individuals and organizations in Ontario. The Niday Quality Audit was conducted under the auspices of the Ontario Perinatal Surveillance System (OPSS). We thank Monica Prince (Prince Computing) who conducted the data analyses and provided input into the final report. We would also like to thank Dr. Ann Sprague for her assistance reviewing the final report, Deshayne Fell for her help in reviewing the manuscript, the auditors for their tireless efforts collecting data at the participating sites across the province and the countless practitioners, data entry personnel and decision makers who provided assistance to make the project possible. References 1. Brown A. Update: building on a foundation to sustain quality data. Toronto (ON): Ministry of Health and Long-Term Care; 2007 Jan. 6. Statistics Canada. Births, estimates, by province and territory [Internet]. Ottawa (ON): Statistics Canada; [modified 2010 Oct 26; cited 2011 Apr 26]. Available from: http://www40.statcan.gc.ca/l01/cst01 /demo04a-eng.htm 7. Freedman G; Health Results Team for Information Management. Building a data quality management framework for Ontario. Toronto (ON): Ministry of Health and Long-term Care; 2006. 8. Canadian Institutes of Health Research; Natural Sciences and Engineering Research Council of Canada; Social Sciences and Humanities Research Council of Canada. Tri-council policy statement: ethical conduct for research involving humans. Ottawa (ON): 2010. 9. Iron K, Manuel DG. Quality assessment of administrative data (QuAAD): an opportunity for enhancing Ontario’s health data. ICES Investigative Report. Toronto (ON): Institute for Clinical Evaluative Sciences; 2007 Jul. 10. Hierholzer WJ. Health care data, the epidemiologist’s sand: comments on the quantity and quality of data. Am J Med. 1991;91(3B):21S-26S. 11. Juurlink D, Preyra C, Croxford R, Chong A, Austin P, Tu J, et al. Canadian Institute for Health Information Discharge Abstract Database: a validation study. Toronto (ON): Institute for Clinical Evaluative Sciences; 2006 Jun. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 42 12. Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 4th ed. Oxford (UK): Oxford University Press; 2008. 16. Bedard M, Martin NJ, Krueger P, Brazil K. Assessing reproducibility of data obtained with instruments based on continuous measures. Exp Aging Res. 2000; 26(4):353-65. 17. Portney LG, Watkins MP. Foundations of clinical research: applications to practice. 2nd ed. New Jersey:Prentice Hall; 2000. 18. Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005;85(3):257-68. 19. Bader MD, Ailshire JA, Morenoff JD, House JS. Measurement of the local food environment: a comparison of existing data sources. Am J Epidemiol. 2010;171(5):609-17. 20. Ali AH. An evaluation of perinatal surveillance system in Eastern and Southeastern Ontario. Ottawa (ON): University of Ottawa; 2003. 21. Fair M, Cyr M, Allen AC, Wen SW, Guyon G, MacDonald RC, et al. Validation study for a record linkage of births and infant deaths in Canada. Ottawa (ON): Statistics Canada; 1999 [Statistics Canada, Catalogue No.: 840013XIE]. 22. Fair M, Cyr M, Allen AC, Wen SW, Guyon G, MacDonald RC. An assessment of the validity of a computer system for probabilistic record linkage of birth and infant death records in Canada. The Fetal and Infant Health Study Group. Chronic Dis Can. 2000;21(1):8-13. 23. Joseph KS, Fahey J. Validation of perinatal data in the Discharge Abstract Database of the Canadian Institute for Health Information. Chronic Dis Can. 2009;9(3):96-100. Identifying potentially eligible subjects for research: paper-based logs versus the hospital administrative database L. A. Magee, MD (1,2,3,4); K. Massey MSc (2); P. von Dadelszen, DPhil (2,3,4); M. Fazio, BSc (2); B. Payne, BSc (2); R. Liston, MB ChB (2,4) This article has been peer reviewed. Abstract Introduction: The Canadian Perinatal Network (CPN) is a national database focused on threatened very pre-term birth. Women with one or more conditions most commonly associated with very pre-term birth are included if admitted to a participating tertiary perinatal unit at 22 weeks and 0 days to 28 weeks and 6 days. Methods: At BC Women’s Hospital and Health Centre, we compared traditional paper-based ward logs and a search of the Canadian Institute for Health Information (CIHI) electronic database of inpatient discharges to identify patients. Results: The study identified 244 women potentially eligible for inclusion in the CPN admitted between April and December 2007. Of the 155 eligible women entered into the CPN database, each method identified a similar number of unique records (142 and 147) not ascertained by the other: 10 (6.4%) by CIHI search and 5 (3.2%) by ward log review. However, CIHI search achieved these results after reviewing fewer records (206 vs. 223) in less time (0.67 vs. 13.6 hours for ward logs). Conclusion: Either method is appropriate for identification of potential research subjects using gestational age criteria. Although electronic methods are less time-consuming, they cannot be performed until after the patient is discharged and records and charts are reviewed. Each method’s advantages and disadvantages will dictate use for a specific project. Keywords: subject identification, audit, health survey, hospital records, health records, database Introduction All clinical research studies begin with identifying potentially eligible subjects. Subjects can be identified by reviewing paper-based hospital or other health records designed for clinical purposes and by querying electronic patient databases used for administrative and/or clinical purposes. The Canadian Perinatal Network (CPN) is a national perinatal database of women with threatened very pre-term birth at 220 to 286 weeks’ gestation (22 weeks and 0 days to 28 weeks and 6 days) admitted to Canadian tertiary perinatal units. CPN began collecting data in August 2005, and by August 2009 involved 14 of Canada’s 23 tertiary perinatal units. CPN-eligible patients must be identified for inclusion based on their presentation to one of the participating units with one of the major causes of threatened very pre-term birth. CPN is a continuous quality improvement project with all data collection performed from patient health records. Within our collaborating centres, the question arose as to the best method of identifying potentially eligible women for inclusion in CPN, since different methods are in use in different centres. These are either traditional paper-based admission records and ward logs or the Canadian Institute for Health Information (CIHI) electronic database of inpatient discharges. As a result, we sought to compare the two methods at the largest CPN centre, BC Women’s Hospital and Health Centre in Vancouver. Methods By August 1, 2009, CPN was enrolling patients from 14 of Canada’s 23 tertiary perinatal units from centres in British Columbia (n = 2 centres), the Prairie provinces (n = 4), Ontario (n = 3), Quebec (n = 3) and the Atlantic provinces (n = 2). CPN was approved in each centre as a continuous quality improvement project. Women are included in the CPN if they are admitted to a participating tertiary perinatal unit at 220 to 286 weeks with one or more of the conditions most commonly associated with very pre-term birth: spontaneous Author references: 1. Department of Medicine, Child and Family Research Institute, Vancouver, British Columbia, Canada 2. Department of Obstetrics and Gynaecology, Child and Family Research Institute, Vancouver, British Columbia, Canada 3. School of Population and Public Health, Child and Family Research Institute, Vancouver, British Columbia, Canada 4. Canadian Perinatal Network (CPN) Collaborative Group, Vancouver, British Columbia, Canada Correspondence: Laura A. Magee, BC Women’s Hospital and Health Centre, 4500 Oak Street, Room D213, Vancouver, BC V6H 3N1; Tel.: (604) 875-2424 ext. 6012; Fax: (604) 875-2961; Email: [email protected] 43 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada pre-term labour with contractions, incompetent cervix, prolapsing membranes, pre-term pre-labour rupture of membranes (PPROM), gestational hypertension, intrauterine growth restriction (IUGR), and/or antepartum hemorrhage (APH).* Women are excluded from the CPN if they are monitored for less than 24 hours in a triage area or obstetrical day unit and then sent home without being admitted to hospital. If the woman is admitted to hospital but later discharged, all subsequent re-admissions are recorded in CPN, up to and including her delivery. Data abstractors identify women in one of two ways. First, delivery suite and antenatal ward records contain a patient log that includes the patient’s name, gestational age, admission location, admission date, and depending on the location the hospital record number (delivery suite only). These paper-based data are collected and manually recorded by the nursing staff to administer clinical care and patient flow throughout the hospital. They are handwritten, often in pencil, and sometimes the names are erased or misspelled. In nine CPN centres, these logs are reviewed either in real-time or retrospectively by the CPN data abstractor, using the gestational age criteria of 220 to 286 weeks. In the five other CPN centres, a data abstractor requests a search of the centre’s CIHI data through decision support staff; the query involves gestational age criteria of 220 to 286 weeks alone because admission diagnoses (as opposed to the final diagnoses made after delivery) are not recorded. The search output yields the mother’s hospital identification number, gestational age, admission date, location of inpatient care and chief medical condition determined after delivery. Both approaches yield potentially eligible patients whose medical charts are then reviewed by the CPN site data abstractor who further defines eligibility and, when this is confirmed, abstracts the relevant patient data into the CPN database. Data collection for CPN started at BC Women’s Hospital in August 2005. Initially, the paper-based system of ward logs was used to identify potential Figure 1 Identification of potentially CPN-eligible women entered into the database (N=244) A 38 (15.6%) 185 (75.8%) B 21 (8.7%) (A) 223 cases identified by paper-based log searches; (B) 206 cases identified by querying the CIHI database. Abbreviations: CIHI, Canadian Institute for Health Information; CPN, Canadian Perinatal Network. subjects, and copies of these records were kept on file until the medical records of all potentially eligible women had been reviewed. Ward logs were obtained from BC Women’s labour and delivery suite, antepartum unit and four postpartum units. In January 2008, subjects started to be identified through an electronic search of the CIHI database for gestational ages 220 to 286 weeks. This initial search was done back to January 2007, creating an overlap in identification methods for the period between April 1, 2007, and December 31, 2007 (the period for which ward logs had still been retained). For this period of overlap, the data abstractor reviewed the list of potential eligible subjects identified by CIHI to identify other potentially eligible women who may have been missed by the patient logs. In July 2009, the patient ward logs available for the period between April 1, 2007, and December 31, 2007, were compared with a corresponding CIHI database search of locally retained data sent to CIHI by the hospital for gestational ages 220 to 286 weeks by a single reviewer who was not aware of which women were actually eligible and enrolled in CPN. We sought to determine the accuracy of paper-based versus electronic search methods of subject identification, as well as the time requirements for each approach, with results expressed descriptively as N (%). Results From April 1, 2007, until December 1, 2007, a total of 244 women were identified as potentially eligible for enrolment in CPN at BC Women’s Hospital based on gestational age criteria (220 – 286 weeks). Figure 1 shows that 185 (75.8%) women were identified by both the paper-based ward log review and the CIHI database output. Each method also identified a small number of women who were not identified by the other method: 38/244 (15.6%) for ward logs and 21/244 (8.7%) for CIHI. Review of the ward logs revealed missing or incorrect information such as surname spelling errors (confirmed on subsequent chart reviews) in 11/223 (4.9%) records, This prevented the data abstractor from further tracking the patient if no other identifiers were present, such as a hospital identification number, which is recorded routinely only by the delivery suite at the BC Women’s Hospital. From April 1, 2007, until December 1, 2007, records for 155 women were entered into the CPN database (at the BC Women’s Hospital site) after manual review of their health records confirmed their eligibility. Figure 2 shows that 137/155 (88.4%) were identified by both the paper-based and electronic database search methods. Similar numbers of women were identified by only one of the two methods: ward logs captured 142/155 of the eligible women (91.6%) including 5 women (3.2%) who * For definitions of indicator conditions and maternal and perinatal outcomes see http://www.cpn-rpc.org/doc/Appendix1_JOGC_20100726.pdf. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 44 the CIHI had missed, and the CIHI data identified 147 of the eligible women (94.8%) including 10 women (6.4%) who were missed during the ward log review. There were also three women (1.9%) who were included in CPN but were neither identified by ward log review nor by CIHI search; these must have been identified by other means, such as word of mouth. It took 13.6 hours to review the paper-based ward logs (i.e. 8 hours to search through the labour and delivery suite logs, 3.8 hours for antepartum unit logs and 1.8 hours for postpartum ward logs). These records had already been photocopied and were assembled and on file, so the actual time required to use the paper-based ward logs for patient selection would be longer. In contrast, the Decision Support Analyst took 0.67 hours to perform an electronic search of the Hospital CIHI data (0.50 hours to set up the initial query and 0.17 hours to run the initial query and each of any subsequent queries and forward the information to the CPN data abstractor. Discussion The CPN uses two major methods to identify patients: review of paper-based ward logs and electronic search of the Hospital CIHI administrative databases, using gestational age-based criteria. The results of our analysis at BC Women’s Hospital and Health Centre, the largest CPN site, showed that both of these approaches identifies the vast majority (88%) of eligible women. The CIHI search identified a further 6.4% of unique records that were not identified by the ward logs, while a search of ward logs identified a further 3.2% that were not identified by the CIHI search. The CIHI search took substantially less time (0.67 hours, which included the initial query set-up, versus at least 13.6 hours for the paper-based ward logs because this estimate did not reflect the time taken to collect and photocopy the ward logs). Review of ward logs has the advantage that it can be done daily, which permits prospective identification of patients. Conversely, a limitation of ward logs is missing or incorrect data (e.g. incorrect spelling of family name, wrong gestational age), which is not surprising as these logs are not intended for research purposes but to plan nursing assignments and manage admissions and discharges. Ward logs may also be difficult to double-check as a result of illegible hand-writing; it is possible that this is the reason for the three entries in the CPN database that were neither identified by CIHI search nor by ward log search. Such an omission may occur within a single shift, when a name is written in pencil and then is removed again, leaving no permanent record. Further, collecting these records, particularly from multiple locations, is time-consuming. Electronic search of hospital administrative data has the advantage of being efficient and reproducible. It can perform more complex searches using structured query Figure 2 CPN-eligible women entered into the database (N=155) A 5 (3.2%) 137 (88.4%) B (A)142 cases identified by paper-based log searches; (B) 147 cases identified by querying the CIHI database. 45 Limitations There are potential limitations to our study. The abstractor who performed this comparison of ascertainment methods was not biased by the initial eligibility assessment, as he did not do the initial review and CPN data entry; however, we were not able to measure inter-rater reliability. Our project relates to using gestational age criteria because neither ward logs nor CIHI data have additional admission diagnoses. However, the accuracy of using CIHI data might be different if additional relevant CIHI terms were available for another project. On the other hand, ward logs are very basic with regard to the information that they contain. Also, additional criteria for review of ward logs and/or CIHI searches may have yielded different results. Conclusion 10 (6.4%) Abbreviations: CIHI, Canadian Institute for Health Information; CPN, Canadian Perinatal Network. language (depending on the clinical question and available data fields).1 It also has the potential to search actual clinical records with increasing use of an electronic health record based on standardized language.2 A limitation is miscoding, which is least likely to occur when basic terms (like “gestational age”) are used.3 The major limitation of this approach is that it cannot be done prospectively or in near real-time. Data are available only after the patient has been discharged and charts have been reviewed and abstracted in the Health Records department, which may take months in some institutions. As such, this method would not be feasible for researchers who need to identify women at or shortly after admission to hospital. Our study suggests that using gestational age-based criteria and either paper-based ward logs or electronic searches of hospital CIHI administrative database are both reasonably accurate methods of identifying potential subjects for clinical audit. Each method has its advantages and disadvantages, but database approaches are far less time-consuming, though they cannot be performed in or near real-time but only until after the patient has been discharged and information is abstracted from the ward logs. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Acknowledgements This study was supported by a grant from the Canadian Institutes of Health Research. Thank you to the CPN site coordinators and previous CPN National Coordinators, D. Chaplain and T. Morris. References 1. Jamison DC. Structured Query Language (SQL) fundamentals. Curr Protoc Bioinformatics. 2003 Feb;Chapter 9:Unit9.2. 2. Massey KA, Ansermino JM, von Dadelszen P, Morris TJ, Liston RM, Magee LA. What is SNOMED CT and why should the ISSHP care? Hypertens Pregnancy. 2009 Feb;28(1):119-21. 3. Schulz EB, Barrett JW, Price C. Read Code quality assurance: from simple syntax to semantic stability. J Am Med Inform Assoc. 1998 Jul;5(4):337-46. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 46 Research methods of the Youth Smoking Survey (YSS) T. Elton-Marshall, PhD (1,2); S. T. Leatherdale, PhD (2); S. R. Manske, EdD (1); K. Wong, MSc (1); R. Ahmed, PhD (1,3); R. Burkhalter, MMath (1) This article has been peer reviewed. Abstract This paper describes the survey development, design and data collection protocol for the 2008/2009 Youth Smoking Survey (YSS) and the changes to the YSS survey and protocols across the 5 survey cycles (1994, 2002, 2004/2005, 2006/2007, 2008/2009). Canada’s Youth Smoking Survey is a nationally representative school-based survey of students (grades 6 to 12 in 2008/2009) from randomly sampled public and private schools in the ten provinces. The main objective of the YSS is to provide benchmark data on national smoking prevalence rates for youth. Key features of the 2008/2009 YSS include consistent measures across survey cycles, a survey team of researchers and non-governmental organizations, a link to school and student level measures, provision of tailored feedback reports to schools and publicly available datasets. Keywords: youth, smoking behaviour, Canadian Youth Smoking Survey, survey cycles, questionnaires Introduction Nationally representative surveys of youth smoking behaviour are necessary to understand the social, regulatory, educational and commercial factors that influence smoking; to provide evidence for tobacco control policies and programs; and to monitor tobacco consumption in Canada.1 The Youth Smoking Survey (YSS) is the only school-based national survey of youth smoking in Canada. The YSS is a cross-sectional classroom-based survey of a representative sample of schools in the 10 Canadian provinces. When first administered in 1994, it was the largest and most comprehensive survey on youth smoking behaviour since 1979 for students in grades 5 to 9. To date, five survey cycles have been conducted (1994, 2002, 2004/2005, 2006/2007, 2008/2009) to monitor changes over time. In 2006/2007, the YSS survey was extended beyond grade 9 to include all other grades of secondary school students (i.e. grades 10 to 12 in most provinces and in Quebec, Secondaire IV to V). The population coverage for YSS 2008/2009 was similar to the YSS 2006/2007 except that grade 5 students were excluded due to the very low smoking rate in this age group. The YSS is undertaken with the cooperation, support and funding of the Controlled Substances and Tobacco Directorate, Health Canada. The research team is pan-Canadian, interdisciplinary, and from university and non-governmental organizations across the country. The main objective of the YSS is to provide comparable benchmark data on national and provincial prevalence rates for youth every two years to guide policy and practice decisions. In addition, it provides a unique opportunity to advance our knowledge of the psychosocial correlates of smoking behaviour, including initiation and cessation. It can help examine individual differences in the influence of tobacco marketing, purchasing controls and other policy initiatives. The YSS offers a detailed snapshot of how youth buy or get cigarettes and of smoking behaviours, and the effects of continued tobacco marketing. This information is critical to assessing the need for increased legislative controls on tobacco and bolstering public support for these policy options. Interventions directed at children and youth are easy for legislators and the populace to support and often encourage tobacco use reduction in adults as well. Without this type of monitoring, we cannot gauge the effectiveness of our prevention efforts. This paper describes the survey development, design, and data collection protocol for the 2008/2009 YSS and highlights changes to this cycle relative to the previous four. Additional information on the design, measures and protocols of this and previous cycles of the YSS are available online.* Methods 2008/2009 YSS development A pan-Canadian consortium of university and non-governmental organizations implemented the 2008/2009 YSS. Members of the Youth Health Team at the Propel Centre for Population Health Impact at the University of Waterloo (Ontario) provided central leadership, while members from the other nine provinces provided leadership in their respective provinces. Members developed survey content during teleconferences. Those who could not participate in the scheduled meetings were asked to provide input prior to the teleconference. This * www.yss.uwaterloo.ca. Author references: 1. Propel Centre for Population Health Impact, Canadian Cancer Society / University of Waterloo, Waterloo, Ontario, Canada 2. School of Public Health and Health Systems, University of Waterloo, Waterloo, Ontario, Canada 3. Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, Ontario, Canada Correspondence: Scott T. Leatherdale, School of Public Health and Health Systems, Faculty of Applied Health Sciences, University of Waterloo, 200 University Ave. West, Waterloo ON N2L 3G1; Tel.: (519) 888-4567 ext 37812; Fax: (519) 886-6424; Email: [email protected] 47 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada approach allowed provincial stakeholders and the federal government to ensure the survey content included measures relevant to each jurisdiction. Content meetings ensured that core items (those required to compute smoking prevalence rates and derive other key, comparable variables) were retained. Questions added to the existing survey were those deemed higher in priority particularly if they were relevant to active policy agendas. The consortium made consensus decisions about which questions to include in the survey after discussing the merit of all survey questions during team teleconferences. Each iteration of the YSS allows for a few new items; however, for every addition, about the same number is removed to keep the questionnaire the same length. Those items that tend to appear every other cycle are considered “periodic.” Some items, known as “deleted items,” are phased out completely if the issue/ question is no longer relevant. While consistent content permits monitoring of trends over time, introducing new items permits identifying new trends that need to be monitored (see the 2008 YSS user guide for a list of survey items by cycle).2 Several key considerations guided the development of content for the 2008/2009 YSS: • Comparability – Core items were kept consistent to allow for comparisons between years. • Responsiveness – To meet data users’ needs, those responsible for federal and provincial tobacco strategies, provincial collaborators and tobacco control advocates contributed topics/items for consideration by the content team. • Relevance – To ensure value-added for participating schools, education-relevant items enhanced school-level feedback reports. • Feasibility – To meet the criterion of being able to complete the survey in a single class period, the length of the questionnaire was restricted. Prior to implementation, the survey questionnaire was pilot tested (in both French and English). During the two-hour pilot-testing sessions, students representing smokers and non-smokers from all grades completed the questionnaire independently and were encouraged to write comments/ questions while doing so. Respondents then participated in a 75-minute focus group discussion in their first language led by a moderator using a pre-developed survey guide. The moderator explored students’ comprehension of the survey questions (with particular focus on all new questions), the logic and order of the questions, and overall flow of the questionnaire. The objectives of the pilot-testing sessions were to: (1) assess the length of time required to complete the survey; (2) probe students’ comprehension of the survey questions (with particular focus on all new questions); and (3) test the logic and order of the questions, including overall flow of the survey instrument. Changes to the survey were based on the feedback obtained in these sessions. Health Canada and the implementation team jointly decided on questionnaire revisions based on these pilot results. Many of the items that have been used in other youth smoking surveys (e.g. Global Youth Tobacco Survey,3 Ontario Student Drug Use and Health Survey4) have been found to be reliable (e.g. current alcohol, marijuana, and tobacco use questions in the Youth Risk Behaviour Survey)5,6 and have been validated in other studies (e.g. assessing attitudes towards smoking, smoking intentions).7 All protocols and materials, including the final survey instrument, received ethics approval from the University of Waterloo Office of Research Ethics and local institutional review boards where required (e.g. in some cases, from two additional levels: the provincial host institution and the school board). Survey measures Core measures. To be consistent, “core” survey measures remain the same across all survey years. These include the measures used to define the smoking status of each respondent according to Health Canada definitions, measures of key prevention indicators such as susceptibility to future Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 48 smoking, age of initiation and amount smoked, and key demographic variables. The core outcomes measured in the YSS are susceptibility to future smoking among never smokers and smoking status. The validated algorithm of Pierce et al. was used to measure susceptibility to future smoking among never smokers (those who have not smoked even a few puffs of a cigarette).8 Susceptibility was determined from responses on a 4-point Likert scale to the following questions: “Do you think in the future you might try smoking cigarettes?”; “If one of your best friends was to offer you a cigarette, would you smoke it?” and “At any time during the next year do you think you will smoke a cigarette?” Never smokers who answered “definitely not” to all three questions were considered non-susceptible; they were considered susceptible to future smoking if they responded positively to at least one of the questions. Smoking status was determined by asking respondents if they had ever tried a cigarette (even just a few puffs), if they had ever smoked a whole cigarette, if they had ever smoked 100 or more whole cigarettes in their lifetime, and on how many of the last 30 days they had smoked one or more cigarettes. Consistent with Health Canada’s operational definitions of smoking status for the YSS,9 respondents were then grouped into the following eight categories: daily smoker (smoked at least 100 cigarettes and currently smokes cigarettes every day); occasional smoker (smoked at least 100 cigarettes and currently smokes cigarettes but not every day); former smoker (smoked at least 100 cigarettes but had not smoked in the last 30 days); experimental smoker (smoked in the last 30 days but had not smoked at least 100 cigarettes); past experimental smoker (had smoked a whole cigarette but had not smoked in the last 30 days and had not smoked at least 100 cigarettes); puffer (had tried smoking but has not smoked a whole cigarette) and never tried (never tried a cigarette, not even a few puffs). Non-core questions. Non-core questions provided information on such issues as where and how youth obtained cigarettes, exposure to second-hand smoke, awareness of health risks due to smoking, and attitudes and beliefs and related health behaviours. Answers to these questions help understand smoking behaviour and uptake among youth, as well as other associated behaviours (e.g. watching television, playing video games). (See Appendix A of the 2008 microdata file to see a comprehensive list of questions and the survey cycles in which these questions appeared).2 Skip patterns. The youth questionnaire was intentionally designed with no respondent-use skip patterns to avoid identifying smokers by rate of survey completion during the classroom session. Thus all smoking behaviour items included a response option such as “I do not smoke.” However, due to the logical flow of the questions, a number of questions were extraneous based on the answer to a previous question. In these cases, a skip pattern was imposed within the operational definitions for appropriate measures within the public use metafile (PUMF), the de-identified dataset available to researchers. If a question could be skipped within the structure of the questionnaire, it was coded as 96 or 996 or 9996 within the PUMF dataset. For example, a smoker would still be asked questions about susceptibility to smoking but the responses for those questions would be coded as a “valid skip” and would be excluded from the analyses associated with smoking susceptibility. Provision of school feedback reports to schools Starting with the 2004/2005 cycle, the YSS used the School Health Action, Planning and Evaluation System (SHAPES) for school-based data collection. Thus each participating school received a school-specific feedback report and executive summary within 10 weeks of data collection. This report provides customized information including smoking rates and other behavioural (e.g. time spent reading) and environmental information (e.g. smoking on school property) specific to the school. As a supplement to the YSS, information about the school environment (programs, policies and the built environment) was also collected.† † ‡ Sampling design the 2004/2005 and 2006/2007 YSS cycles. The target population for the YSS consisted of all young Canadian residents attending private and publicly funded schools in the 10 Canadian provinces. Those residing in the Yukon, Nunavut and Northwest Territories and those living in institutions or on First Nations reserves were not included in the sampling frame. Young persons who were attending special schools (e.g. schools for visually and hearing-impaired) or schools located on military bases were also excluded from the sampling frame. In Ontario, the design of the 2008/2009 cycle included a third health region stratum, Greater Toronto Area (GTA). The GTA health region stratum acknowledged the size of the GTA and the importance of being able to capture schools from the GTA even if there were refusals from the larger school boards in the city of Toronto. The YSS team at the Propel Centre obtained a comprehensive list of all schools in each province via provincial Department of Education websites. The sampling for the YSS was based on a stratified multistage design. Sampling was stratified according to health region smoking rate and type of school (elementary or secondary). In Stage 1, the Canadian Community Health Survey (CCHS) was used to calculate the smoking rate among 15 to 19 year olds for each health region. The school lists obtained from the provincial Departments of Education for each of the 10 provinces included enrolment data by grade for each school. Using this list, the total eligible grade enrolment in a health region was used as a weight to compute the median smoking rate for each province. Each school’s six-digit postal code was used to identify the health region in which it was located. Schools were then categorized as “low” or “high” smoking rate stratum based on the smoking rate in their health region compared to the median (where greater than or equal to the median was categorized as “high”). In Stage 2, schools were stratified into elementary or secondary school strata (calculated based on whether there was a higher enrolment of students in grades 6 to 8 or 9 to 12). Elementary and secondary schools were sampled on a 2:1 ratio due to the smaller enrolment sizes of the elementary schools. Schools were also over-sampled in each province based on the provincial school recruitment rate from Lastly, sampling of private schools was based on a simple random sample of private schools in each province. The number of schools originally selected was roughly proportional to the number of students enrolled in private schools in that province as compared to the total in public schools. The sampling design is constructed to provide a representative sample of youth in all provinces in Canada. In the 2008/2009 cycle, the school board response rate was 84% (the number of school boards that agreed to participate/ the number of school boards that were approached); the school level response rate was 59% (the number of schools that agreed to participate/the number of schools that were approached); and the student level response rate was 73.2% (based on the number of completed surveys/the number of eligible students; students who were absent during the data collection were counted as a non-response). Survey protocol In all provinces, YSS site coordinators contacted school boards prior to approaching schools. Private schools were approached directly because there is no governing board to review research requests for these schools. School boards were typically contacted via a formal board-specific application or a standard board recruitment package that included a school invitation letter, a project brochure, a sample student survey, sample parent information and permission materials, and a template school feedback report.‡ Provincial site coordinators made follow-up calls to the school board to answer any questions and, ideally, obtain board More information about the SHAPES, including sample reports, can be found at www.shapes.uwaterloo.ca. For sample documents, e.g. surveys, feedback report, etc., see www.yss.uwaterloo.ca/recruitment. 49 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada permission to recruit schools. Once a school board was successfully recruited, the schools within that school board were approached via a school recruitment package and follow-up phone calls. The contents of the school recruitment packages were the same for both boards and schools. Only when the school had agreed to participate in the YSS was the survey implemented with eligible students in that school. Within each participating school, all students in the eligible survey grades (6 to 12) were requested to complete the survey. Active parental permission was required by the school or board for 62% of grade 6 to 8 classes (n = 913) and 19% of grade 9 to 12 classes (n = 372). Students in eligible classrooms took home information letters describing survey details. Active permission protocols required signed parental and child permission forms for the child to receive and complete a survey. In 81% of secondary school classes (n = 1631), passive permission protocols were used to reduce the burden on schools and improve response rates. In this procedure, the school mailed an information letter home to parents that detailed survey procedures, and asked parents to call a toll-free number or inform the school if they did not want their child to participate. Students whose parents objected were put on a “no permission” list and did not receive a survey on the day of data collection. All other students received a survey to complete. Regardless of whether parents provided permission, students were able to decline participation on the day of data collection. Provincial site coordinators worked with a school contact to arrange data collection at each school. On the day of data collection, teachers administered the survey using standardized protocols during a designated class period. To ensure confidentiality and therefore encourage honest responses, teachers were asked to avoid circulating among the students. Students were also required to place their completed survey in an envelope and seal this envelope before it was collected by a student in the classroom. When parents as well as students were surveyed, active consent was required, and a tear-off sheet with the student’s name was attached to the front of the survey. Students removed the tear-off sheet. A serial code on both the tear-off sheet and the student survey enabled linkage for survey cycles that included a parent interview to be linked to the student responses. The information containing the student’s identification and responses were removed from all public datasets and only those directly related to the research had access to any identifying information. On average, the survey took 30 to 40 minutes to complete. A data collector was on site at the school throughout the data collection period and available to answer respondent questions and collect the completed student surveys. Data management Surveys were machine scanned using Optical Mark Reading (OMR) technology. Quality control measures (e.g. visual scanning, OMR scanning twice to find discrepancies) were used to ensure accuracy of the scanned data. An online survey implementation system (OSIS) permitted central management of recruitment, implementation, analysis and feedback processes. Survey weights Survey weights were created to “weight” the data to be representative of the general population of Canadian youth in school. The survey weights were developed in two stages. In the first stage, a weight (W1) was created to account for the school selection within health region and school strata. A second weight (W2) was then calculated to adjust for student non-response. The weights were then calibrated to the provincial gender and grade distribution so that the total of the survey weights by gender, grade and province would equal the actual enrolments in those groups. Finally, bootstrap weights for each province (to estimate sampling error) were generated. Evolution of the YSS The protocols described were used to implement the 2008/2009 YSS cycle. One of the strengths of the YSS has been Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 50 its consistent protocols, which allow comparisons over cycles. However, there have been slight modifications to the sampling and protocols in each cycle based on experience in previous cycles; these modifications were made to improve student recruitment and survey completion rates, and to reduce the burden on participating schools. The following section describes some of the significant differences in the YSS over the various cycles. Changes to survey administration. One of the most significant changes to the YSS occurred in 2004/2005 when the survey administration shifted from Statistics Canada to the University of Waterloo. In 1994 and 2002, the YSS content was developed by Health Canada’s Office of Tobacco Control and data were collected by Statistics Canada. As previously noted, the University of Waterloo’s Propel Centre for Population Impact (formerly the Population Health Research Group and the Centre for Behavioural Research and Program Evaluation) has provided central leadership since 2004/2005. Changes to the survey. Table 1 summarizes the differences in the survey over time. Until 2006/2007, the sample included grades 5 to 9 only. In 1994, all students in grades 5 to 9 responded to the same survey. In 2002 and 2004/2005, students in grades 7 to 9 answered additional questions about alcohol and drug use. In 2006/2007, students in grades 7 to 12 were randomly assigned to receive one of two versions of the survey. While the majority of the questions were the same in both versions, including those that related to alcohol and drug use, some different questions were added to each (e.g. in one version there were questions about smoking on school property whereas another version had questions about beliefs about the harmful effects of smoking). All other students (grades 5 and 6) received a survey with no questions on alcohol and drug use. Because there were two different versions of the survey in this cycle, there were two survey weights calculated for this dataset and two User Guides to facilitate use of the dataset. Table 1 Features of the Youth Smoking Survey by survey cycle Survey cycle Survey dates Target population, grades Sample size (n) Changes to the survey protocol 1994 Sep–Nov 1994 5–9 14 270 2002 Oct–Dec 2002 5–9 19 018 Students in grades 7–9 answered additional questions about alcohol and drug use 2004/2005 Feb–Jun 2005 5–9 29 243 Adoption of SHAPES (School Health Action Planning and Evaluation System) Computer-generated feedback reports delivered to schools Surveys machine-scanned using Optical Mark Read (OMR) technology 2006/2007 Nov 2006– Jun 2007 5–12 71 003 Addition of grades 10–12 Collaboration with Healthy New Brunswick en santé, Project Impact, and the Canadian School Smoking Policy Survey The student survey data were collected using three instruments: • Module A: 66 questions administered to all students in grades 5–6. Did not include drug and alcohol question Students in grades 7–12 completed either Module B1 or B2: • Module B1: 76 questions including some questions from Module A, some new questions, and drug and alcohol questions • Module B2: 84 questions including questions from Module A, some new questions, and drug and alcohol questions In New Brunswick, data were collected to support the Healthy New Brunswick en santé project (data on smoking using YSS, healthy eating, physical activity, and mental fitness) Census of schools in New Brunswick In New Brunswick, 50% of students in grades 5–6 completed the YSS Module A, 25% of students completed a Physical Activity Module and 25% completed a Healthy Eating Module. Within each class in grades 7–12, 25% of students completed the YSS Module B1, 25% of students completed the YSS Module B2, 25% of students completed a Physical Activity Module and 25% of students completed a Healthy Eating Module 2008/2009 Dec 2008– Jun 2009 6–12 51 922 Grade 5 students no longer included in the survey The student survey data were collected using two instruments: • Module A: 57 questions administered to students in grade 6. Module A did not include drug and alcohol questions • Module B: 65 questions administered to students in grades 7 through 12. Items included all questions from Module A and drug and alcohol questions Collaboration in PEI with the Comprehensive School Health Research Group supporting the SHAPES-PEI project, which collected data on smoking (YSS), healthy eating, physical activity and mental fitness. Among grade 5 students, 50% completed a Healthy Eating Module and 50% completed a Physical Activity Module. Among grade 6 students, 50% completed the YSS Module A, 25% completed the Healthy Eating module and 25% completed the Physical Activity module. In grades 7–12 in each school, 50% of the students completed YSS module B and 50% completed the SHAPES module (all questions) In 2008/2009, grade 5 students were no longer included in the survey, primarily because of the low prevalence of smoking among students in this grade and the challenges of having students in this grade to complete the survey in the time allotted. Grade 6 students completed the survey without the alcohol and drug use questions whereas those in grades 7 to 12 completed a survey that included alcohol and drug use questions. Collaboration. Whenever possible, YSS data collection was coordinated with other data collections taking place at the same time. In 2006/2007, YSS collaborated with the University of New Brunswick’s Health & Education Research Group (HERG) and with the Comprehensive School Health Research Group in Prince Edward Island to implement their provincial surveys in 2008/2009 (NB Wellness Survey and SHAPES-PEI, respectively). Both initiatives collected data on smoking (YSS), healthy eating (HE), physical activity (PA), and mental fitness (MF) from students in grades 5 to 12 (grades 6 to 12 for NB Wellness with the exception of YSS-sampled schools, which were grades 5 to 12). The data included a census of eligible schools in the respective provinces. The YSS dataset does not include any data collected from the 51 NB Wellness or SHAPES-PEI additional modules, but the dataset does include the additional students who responded to the YSS. The data collection procedures therefore varied slightly for NB and PEI. Table 1 summarizes these differences in data collection. Changes to sampling design In 1994, the sample design consisted of a two-stage stratified clustered design in which schools were the primary sampling units and classes were the secondary units. There were two levels of stratification. Each province was the main stratum and Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada there was an implicit stratification by grade. The school sample was selected systema tically with probability proportional to school size (the total number of students for each grade). Classes within schools were randomly selected and all students in a selected class were included in the final sample. In 2002, the sample design featured three levels of stratification. Each province was the main stratum and there was an implicit stratification by grade. Schools were also stratified by census metropolitan area (CMA) versus non-CMA, with additional strata in Quebec (Montréal) and Ontario (Toronto). The sample was then selected in each stratum independently, meaning that some schools could be selected more than once for different grades. Classes were randomly selected from the schools that were recruited. In 2004/2005, the sampling was conducted in two stages. In stage 1, school boards were sampled within each province. The Canadian Community Health Survey (CCHS) was used to estimate the current smoking rate at the level of health region. Estimated adult smoking rates were calculated for each school board and the school boards were ranked and categorized as “upper stratum” or “lower stratum.” In stage 2, schools were sampled from the list of selected school boards. School boards were selected based on their adult smoking rate. Within each selected school board, schools were stratified into two strata: senior strata (students in senior elementary or high school grades) or junior strata (students in a school with grades 5, 6, 5–6, 5–7, and 6–7). Where possible, there was an over-selection of junior stratum schools. All eligible grades within a school were selected to participate, rather than just a random selection of classes within a school. The sample design in 2006/2007 was the same as the design described for the 2008/2009 survey cycle with a few small exceptions. The smoking rate calculated for the province and health region was based on adult smoking rates, and there § was no separate stratum for the Greater Toronto Area. Again, all classes in eligible grades in selected schools were surveyed. Discussion The Youth Smoking Survey is a nationally representative school-based survey of youth in Canada. The YSS was designed to provide both national (excluding Yukon, Northwest Territories and Nunavut) and provincial estimates of smoking prevalence, as well as surveillance of tobacco-related knowledge, attitudes and behaviours of young people in Canada. However, the YSS is more than a surveillance tool. It was designed to assess and help develop public education programs and policies for tobacco control. With the integration of the YSS with the SHAPES, the YSS is even more capable of integrating tobacco control policy and practice and monitoring the effectiveness of tobacco control strategies through the school-specific feedback reports. There are several unique features of the 2008/2009 YSS when compared to other Canadian surveys: • The core measures used in the YSS are maintained over survey cycles. This allows monitoring tobacco use over time and evaluation of tobacco control policies/programs (using a quasi-experimental design, comparing survey measures before and after a tobacco control policy or program is implemented). These core measures are also consistent with other existing surveys to allow comparisons between groups.2 • Governmental and non-governmental organizations as well as researchers make up the YSS consortium. These individuals develop survey questions based on their knowledge of priority tobacco control topics. The questions therefore reflect topics that are timely and regionally relevant and that can influence policy development and evaluation. • Through the SHAPES model, there is the opportunity to link with school-level data (not part of the PUMF distributed provincially and to universities) and student level data. These data are http://www.propel.uwaterloo.ca/index.cfm?section=28&page=377. ** http://www.statcan.ca/english/Dli/dli.htm. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 52 collected in parallel to the YSS although not as a core part of the YSS. The data can therefore be used to understand the school context and evaluate school-based prevention initiatives. Research has demonstrated that interventions are sometimes effective in one setting but not another.10 An intervention may therefore be effective in one school but not another, and it is therefore important to incorporate the school level in data analyses. • Tailored feedback reports are given to schools. This information provides stakeholders at the schools with locally relevant real world data to inform prevention planning. Schools are empowered to take ownership of their school policies to protect the health of their students rather than relying on outside regulatory bodies. • Publicly available datasets of the 2008/2009 YSS PUMF have been sent to each provincial government and Canadian university research library. The dataset can be requested through the Propel Centre’s Population Health Data Repository,§ which also has publicly available raw data, and Statistics Canada Data Liberation Initiative (DLI).** Both the Propel Centre and Health Canada also have summary tables from each survey year available on their websites. The YSS has been used to guide tobacco control policies and programs nationally. For instance, the 2008/2009 YSS data were instrumental in prompting the federal government to amend the Tobacco Act in 2009 as part of Bill C-32 to prohibit the use of flavour additives in cigars and cigarillos.11 YSS data have also been used to inform provincial tobacco control policies and strategies. For instance, during the 2010 renewal of the Smoke-Free Ontario Strategy, YSS data played a key role in informing the policy recommendations in Chapter 5 of the new guide for comprehensive tobacco control in Ontario.12 The YSS has also been used by researchers to understand tobacco use among youth in Canada and to identify and inform future tobacco control priorities including tobacco use among off-reserve Aboriginal youth in Canada,13 contraband cigarettes use,14 bidi and hookah use,15 alcohol and illicit substance use,16-19 cigarette brand preferences and price,20 taxation,21 second-hand smoke exposure,22,23 cigarette access,24 school policies and smoking,25 socialization towards smoking26,27 and smoking among adolescent girls.28 The YSS has expanded to collect relevant information on other risk behaviours (physical activity, obesity, healthy eating). This data will be used to make future policy and programming decisions regarding other health policies in addition to tobacco control. The 2010/2011 cycle of the YSS is currently in progress, and we hope that researchers and policymakers will continue to use this important dataset to understand tobacco use and other risk behaviours among youth in Canada. Acknowledgements The authors would like to thank the Propel Centre for Population Health Impact for providing support for this project. Dr. Leatherdale is a Cancer Care Ontario Research Chair in Population Studies. The 2008/2009 Youth Smoking Survey is a product of a pan-Canadian capacitybuilding project that includes Canadian researchers from all provinces and provides training opportunities for university students at all levels. Production of this paper has been made possible through a financial contribution from Health Canada. The views expressed herein do not necessarily represent the views of Health Canada. The authors recognize the key role that Murray Kaiserman, now retired Director in Health Canada’s Tobacco Control Programme and the Controlled Substances and Tobacco Directorate, has played in the initiation and continued funding of the YSS. References 1. Stephens T, Morin M, editors. Youth Smoking Survey, 1994: Technical Report. Ottawa (ON): Minister of Supply and Services Canada; 1996. 2. Youth Smoking Survey (YSS): 2008-09 Y SS Microdata User Guide [Internet]. Waterloo (ON): Propel Centre for Population Health Impact; 2009 [cited 2011 April 11]. 53 p. Available from: http: //www.yss.uwaterloo.ca/yss_papers /Documents/yss08_user_guide_EN.pdf 11. Bill C-32: an act to amend the Tobacco Act [Internet]. 2009 Jun 4; revised 2010 Feb 4 [cited 2010 Oct 22], c. 32. Available from: http://www2.parl.gc.ca/Sites/LOP /LegislativeSummaries/Bills_ls.asp?lang =E&ls=c32&source=library _prb&Parl=40&Ses=2 3. Global Youth Tobacco Survey Collaborative Group. Tobacco use among youth: a cross-country comparison. Tobacco Control. 2002;11:252-70. 12. Smoke-Free Ontario Scientific Advisory Committee. Evidence to guide action: comprehensive tobacco control in Ontario. Toronto (ON): Ontario Agency for Health Protection and Promotion; 2010. Chapter 5, Prevention of tobacco use among youth and young adults; p. 75-92. Available from: http://www.oahpp.ca/services/documents /evidence-to-guide-action/Chapter%205 % 2 0 - % 2 0 E v i d e n c e % 2 0 t o % 20 G u i d e %20Action% 20 - % 20 C TC % 20 i n %20Ontario%20SFO-SAC%202010E.pdf 4. Paglia-Boak A, Mann RE, Adlaf EM, Rehm J. Drug use among Ontario students, 1977-2009: Detailed OSDUHS findings. (CAMH Research Document Series No. 27). Toronto (ON): Centre for Addiction and Mental Health; 2009. 5. Brener ND, Kann L, McManus T, Kinchen SA, Sundberg EC, Ross JG. Reliability of the 1999 Youth Risk Behavior Survey Questionnaire. J Adolesc Health. 2002;31:336-42. 6. Brener ND, Collins JL, Kann L, Warren CW, Williams BI. Reliability of the Youth Risk Behavior Survey Questionnaire. Am J Epidemiol. 1995;141(6):575-80. 13. Elton-Marshall T, Leatherdale ST, Burkhalter R. Tobacco, alcohol and illicit drug use among Aboriginal youth living off-reserve: results from the Youth Smoking Survey. CMAJ. 2011;183(8):E480-6. 14. Callaghan RC, Veldhuizen S, Leatherdale S, Murnaghan R, Manske S. Use of contraband cigarettes among adolescent daily smokers in Canada. CMAJ. 2009;181(6-7):384-6. 7. Ford KH, Diamond PM, Kelder SH, Sterling KL, McAlister AL. Validation of scales measuring attitudes, self-efficacy, and intention related to smoking among middle school students. Psychol Addict Behav. 2009;23(2):271-8. 15. Chan WC, Leatherdale ST, Burkhalter R, Ahmed R. Bidi and hookah use among Canadian youth: an examination of data from the 2006 Canadian Youth Smoking Survey. J Adolesc Health. 2011;49(1):102-4. 8. Pierce JP, Choi WS, Gilpin EA, Farkas AJ, Merritt RK. Validation of susceptibility as a predictor of which adolescents take up smoking in the United States. Health Psychol. 1996;15:355-61. 16. Hammond D, Ahmed R, Yang WS, Burkhalter R, Leatherdale S. Illicit substance use among Canadian youth: trends between 2002 and 2008. Can J Public Health. 2011;102:7-12. 9. Summary of Results of the 2008-09 Youth Smoking Survey [Internet]. Ottawa (ON): Health Canada; 2010 [cited 2010 Jul 25]. Available from: http://www.hc-sc.gc.ca /hc-ps/tobac-tabac/research-recherche /stat/_survey-sondage_2008-2009 /result-eng.php 17. Leatherdale ST, Ahmed R. Alcohol, marijuana, and tobacco use among Canadian youth: do we need more multi-substance prevention programming? J Prim Prev. 2010;31(3):99-108. 10. Wang S, Moss JR, Hiller JE. Applicability and transferability of interventions in evidence-based public health. Health Promot Int. 2006;21:76-83. 53 18. Leatherdale ST, Hammond D, Ahmed R. Alcohol, marijuana, and tobacco use patterns among youth in Canada. Cancer Causes and Control. 2008;19:361-9. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 19. Leatherdale ST, Hammond D, Kaiserman M, Ahmed R. Marijuana and tobacco use among young adult smokers in Canada: are they smoking what we think they are smoking? Cancer Causes Control. 2007;18(4):391-7. 28. Seguire M, Chalmers KL. Late adolescent female smoking. J Adv Nurs. 2000;31(6):1422-9. 20. Leatherdale ST, Ahmed R, Barisic A, Murnaghan D, Manske S. Cigarette brand preference as a function of price among smoking youth in Canada: are they smoking premium, discount or native brands? Tob Control. 2009;18(6):466-73. 21. Sen A, Wirjanto T. Estimating the impacts of cigarette taxes on youth smoking participation, initiation, and persistence: empirical evidence from Canada. Health Econ. 2009;1264-80. 22. Leatherdale ST, Ahmed R. Second-hand smoke exposure in homes and in cars among Canadian youth: current prevalence, beliefs about exposure, and changes between 2004 and 2006. Cancer Causes Control. 2009;20(6):855-65. 23. Leatherdale ST, Smith P, Ahmed R. Youth exposure to smoking in the home and in cars: how often does it happen and what do youth think about it? Tob Control. 2008;17(2):86-92. 24. Leatherdale ST, Ahmed R, Vu M. Factors associated with different cigarette access behaviours among underage smoking youth who usually smoke contraband (native) cigarettes. Can J Public Health. 2011;102(2):103-7. 25. Lovato CY, Pullman AW, Halpin P, Zeisser C, Nykiforuk CI, Best F, et al. The influence of school policies onsmoking prevalence among students in grades 5-9, Canada, 2004-2005. Prev Chronic Dis. 2010;7(6):1-10. 26. Nowatzki J, Schultz AS, Griffith EJ. Discrepancies between youth and parent perceptions of their household environment relevant to smoking: a secondary analysis of the 2004/05 Canadian Youth Smoking Survey. Chronic Dis Can. 2010;30(3):78-83. 27. Schultz AS, Nowatzki J, Dunn DA, Griffith EJ. Effects of socialization in the household on youth susceptibility to smoking: a secondary analysis of the 2004/05 Canadian Youth Smoking Survey. Chronic Diseases in Canada. 2010;30(3):71-77. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 54 Self-Monitoring Blood Glucose Workshop I: promoting meaningful dialogue and action at the provincial level M. J. Dunbar, MEd This article has been peer reviewed. Introduction The Nova Scotia Department of Health and Wellness supports a number of provincial programs, including the Diabetes Care Program of Nova Scotia (DCPNS), that function in an advisory capacity to the health department. Committed to ongoing improvement of the health care system and to the promotion of uniform standards throughout the province, these programs bring together experts / working groups to advise the system, recommend service delivery models, establish and monitor approved standards, guide policy and facilitate knowledge transfer/translation and networking in support of best/promising practices. The aim is to improve care and outcomes at the local, district and provincial levels. The development of the DCPNS Self-Monitoring Blood Glucose (SMBG) Decision Tool and the SMBG Workshop and related follow-up work are a cogent example of how a provincial program can quickly mobilize a broad range of experts and front-line health care providers to address an important issue like SMBG. during their national conferences, in 2005 and 2006 respectively. In November 2006, Alberta’s Institute of Health Economics hosted the first Canadian Consensus Conference on Self-Monitoring in Diabetes. National and international speakers presented clinical evidence as well as economic, policy and consumer perspectives. An expert panel assimilated the information and formulated responses to predetermined questions into a consensus document intended for use by all sectors in decision-making around SMBG in Canada.1 Background This consensus work was followed by local, national and international work, including a qualitative study on health care professional views and practices related to SMBG in Nova Scotia;2 recommendations and reports by the Canadian Agency for Drugs and Technologies in Health (CADTH);3,4 costing reports from Ontario’s Institute for Clinical Evaluative Sciences;5 peer-reviewed publications;6-8 workshop presentations9 and the International Diabetes Federation’s guidelines on Self-Monitoring of Blood Glucose in Non-Insulin Treated Type 2 Diabetes,10 among others. “Should all people with diabetes mellitus self-monitor their blood glucose?” This question has received increasing attention in recent years as individuals and the health care system struggle with costs related to testing, the limited evidence in support of testing for some populations, and the realities of using test results for persons with diabetes and their health care providers. This topic is not new. The American and Canadian Diabetes Associations hosted debates on SMBG The DCPNS participated in some of this earlier work. More recently, the Program led discussions to help guide and inform policy and contribute to finding a sustainable, realistic solution to SMBG. Such a solution would help reduce the burden of unnecessary and sometimes wasteful testing in a specified population with diabetes. Details of the SMBG Workshop are presented below, and details of the follow-up work and the SMBG Decision Tool are presented elsewhere.11 Workshop audience and objectives In January 2010, the DCPNS invited a multidisciplinary group of diabetes health care professionals to discuss the Canadian Optimal Medication Prescribing and Utilization Service (COMPUS) recommendations regarding SMBG for non-insulin-treated type 2 diabetes, specifically its use, frequency and application in Nova Scotia. Numerous local and national observers also attended the workshop to gain insight from the discussions (see Table 1). Participants were tasked with the following: • helping to formulate preliminary consensus recommendations, with the help of case-based discussions, on diagnostic strip usage for non-insulin-treated type 2 diabetes mellitus; • identifying potential criteria for “exception status” in SMBG strip requirements for non-insulin-treated type 2 diabetes; and • recommending next steps regarding patient and provider tools, supports and communications. Plenary sessions Four plenary sessions, focusing on the Nova Scotian context, contributed to understanding the evidence underlying the following COMPUS recommendation: “For most adults with type 2 diabetes using oral antidiabetes drugs (without insulin) or no antidiabetes drugs, the routine use of blood glucose test strips for SMBG is not recommended.”3,p5 Author references: Diabetes Care Program of Nova Scotia, Halifax, Nova Scotia, Canada Correspondence: Margaret (Peggy) J. Dunbar, Suite 548 Bethune Building, 1276 South Park Street, Halifax, NS B3H 2Y9; Tel.: (902) 473-3209; Fax: (902) 473-3911; Email: [email protected] 55 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Table 1 Invited participants and observers at the Diabetes Care Program of Nova Scotia (DCPNS) Self-Monitoring Blood Glucose (SMBG) Workshop Invited participants Observers • Diabetes Centre educators from each of Nova Scotia’s nine district health authorities • Canadian Agency for Drugs and Technologies in Health (CADTH) • Nova Scotia division of Canadian Diabetes Association (CDA) ° Registered nurses • Pharmaceutical Services, Nova Scotia Department of Health and Wellness ° Professional dietitians • Drug Evaluation Unit, Capital District Health Authority (CDHA) • Pharmacists • Behaviour Change Institute, CDHA • Physicians • College of Pharmacy, Dalhousie University ° Family physicians • Academic Detailing, Dalhousie University ° Specialist physicians • Pharmacist, First Nations and Inuit Health, Atlantic Region, Health Canada Internists Endocrinologists • Nurse practitioners The plenary sessions (see Table 2) were followed by an exercise that required participants to consider the following: • “What did you hear... what hit home with you?” • “What were the main take-away messages for you?” Six theme areas emerged; these are shown below with brief summary points and/or illustrative quotes. 1. Costs/Wastage • Awareness of escalating costs and the need for fiscal respon sibility: “The potential savings are huge.” 2. Research • Acknowledgment and better understanding of the lack of evidence supporting SMBG and improved outcomes. • Need for more research: “Who benefits from SMBG and in what ways?” 3. Variations in practice • Appreciation of variations in practice among and between diabetes practitioners. • Need for education and programming on how to use, interpret and act on SMBG results. Table 2 Plenary sessions at the Diabetes Care Program of Nova Scotia (DCPNS) Self-Monitoring Blood Glucose (SMBG) Workshop Title Presenter Content Self-Monitoring of Blood Glucose (SMBG): Highlights from CADTH’s Recommendations Denis Bélanger, BSc(Pharm), ACPR, Acting Senior Director, CADTH The first session provided insights into the COMPUS recommendation and the approach used to adopt optimal practice of SMBG. The presentation included an overview of available evidence about the clinical effectiveness and cost effectiveness of SMBG, potential opportunity costs and the key issues that were addressed in the recommendation deliberations. Self-Monitoring of Blood Glucose: The Health Care Professional Perspective Wayne Putnam, MD, Associate Professor, Department of Family Medicine, Dalhousie University This session provided preliminary findings from a qualitative study2 conducted in Nova Scotia “to gain insight into health professionals’ recommendations for, and perceived value of, SMBG in adults with type 2 diabetes who are not using insulin and are in good control (A1C ≤ 7%).” Interviews conducted with diabetes educators, community-based pharmacists and practising clinicians demonstrated variations between and within practice disciplines with regards to the frequency of recommended monitoring, reasons for monitoring, use of results and in the trusted sources of information related to SMBG. Patient and Provider Perspectives on Self-Monitoring of Blood Glucose: Highlights from CADTH’s Focus Groups Denis Bélanger, BSc(Pharm), ACPR, Acting Senior Director, CADTH This session provided an overview of patient and health care professional perspectives as derived from focus groups (Halifax and Ottawa) regarding CADTH’s key messages on the practice of self-monitoring. The presenter shared observations highlighting variations between patients, physicians / nurse practitioners, diabetes educators and pharmacists around why to test, the value of testing and use of results. Individuals with diabetes provided additional perspectives on the advantages and disadvantages of SMBG. Utilization of Blood Glucose Monitoring Strips: Nova Scotia Pharmacare Programs Natalie Borden, BSc(Pharm), Manager, Drug Utilization Review, NS Department of Health and Wellness The final presentation showed the current NS costs for diabetes medications and test strips as well as the number of test strips (and range) being used by the different diabetes treatment types (insulin, oral agents, insulin and oral agents, diet only). Findings from the most recent studies related to this topic5,8,12 were presented, including proposed scenarios for reducing costs of test strips. Abbreviations: CADTH, Canadian Agency for Drugs and Technologies in Health; COMPUS, Canadian Optimal Medication Prescribing and Utilization Service; NS, Nova Scotia; SMBG, self-monitoring blood glucose. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 56 4. Messaging • Information needs to be relayed to persons with diabetes and care providers about the impact of SMBG on outcomes as well as current perceptions and practices. • There is a need for consistent messaging and to refocus patient monitoring on those things that will make a difference in day-to-day management and patient outcomes— food intake, activity/exercise, weight, medication persistence, etc. • Everyone needs to agree on recommendations about who should test and, for those who should test, the frequency of testing. 5. Changes in practice • There is no evidence to support the belief that SMBG is a motivator and results in better outcomes in this population: “We need to rethink SMBG for those that really need it and will benefit. This rethinking will result in a huge shift in practice and how we interact with patients.” 6. Opportunity • Need to change current SMBG guidelines and better understand how SMBG fits within the concept of self-care. Case-based discussions The second half of the workshop focused on case-based discussions and small group work facilitated by clinical experts, Drs. Lynne Harrigan (Internist) and Dale Clayton (Endocrinologist). Cases moved from simple to complex and explored SMBG considerations related to diagnosis, degree of hyperglycemia, type of diabetes treatment, risk of hypoglycemia, and the influences of age, occupation, interest, cognition and motivation. Participants were introduced to a draft SMBG Decision Tool developed by the DCPNS. The draft tool had three focal areas: 1. instructions for how to use and interpret the tool; 2. indications and considerations for SMBG (e.g. safety, planned use of the results by the individual and his/her health care team, and self-management education); and 3. SMBG recommendations (e.g. specific examples of low and high intensity testing with a focus on “time-limited” testing). Participants used the tool as they worked through seven cases studies, as they would be expected to do in practice. According to the participants, “the tool allowed for a more objective look at each individual case and removed emotion and subjectivity from the equation.” It allowed for a focus on patient safety, available evidence, an individual’s interest and capability, and the health care provider’s use of results. In cases for which testing is recommended, the tool also helped participants to determine the intensity of testing required (e.g. low-intensity versus time-limited, high intensity). Following the case studies, participants committed to continuing the dialogue and refining the SMBG Decision Tool by responding to consensus questions and a “Needs and Wants” exercise via email. This feedback will help guide DCPNS and other partners in the development and delivery of resources and programs to move forward a more standardized approach to SMBG in Nova Scotia. Conclusion and next steps Through leadership and partnership, the DCPNS demonstrated the value of addressing the SMBG issue through local dialogue, decision, and provider and patient supports as well as planned, thoughtful dissemination strategies to increase reach into a variety of provider groups. The DCPNS refined the SMBG Decision Tool and worked with its partners and other stakeholders to reach across provider groups to attain consistency in approach and messaging for SMBG in the noninsulin-treated type 2 diabetes population. The results of this continued work are reported in Part II of this article.11 57 Acknowledgements The Diabetes Care Program of Nova Scotia would like to thank all the participants of the Self-Monitoring Blood Glucose Workshop for helping to guide this important work and the observers for their encouragement and support. In particular, we would like to acknowledge Bev Harpell, Brenda Cooke, and Dr. Lynne Harrigan for helping to shape the draft SMBG Decision Tool and the agenda for the SMBG Workshop. Last, but not least, we would like to thank Pam Talbot for her contributions toward the preparation of this manuscript. References 1. Institute of Health Economics. Consensus statement on self-monitoring in diabetes [Internet]. Edmonton (AB): Institute of Health Economics; 2006 Nov 14-16 [cited 2010 Aug 24]. Available from: http://www.ihe.ca /documents/consensus_statement_complete _nov17_0.pdf 2. Latter C, McLean-Veysey P, Dunbar P, Frail D, Sketris I, Putnam W. Self-monitoring of blood glucose: what are health care professionals recommending? Can J Diabetes. 2011;35(1):31-9. 3. Canadian Agency for Drugs and Technologies in Health. Optimal therapy recommendations for the prescribing and use of blood glucose test strips. Optimal Therapy Report – COMPUS [Internet]. Ottawa (ON): CADTH; 2009 Jul [cited 2010 Aug 24]. Available from: http://www .cadth.ca/media/pdf/compus_BGTS_OT _Rec_e.pdf 4. Canadian Agency for Drugs and Technologies in Health. Current practice analysis of health care providers and patients on self-monitoring of blood glucose. Optimal Therapy Report – COMPUS [Internet]. Ottawa (ON): CADTH; 2009 Mar [cited 2010 Aug 24]. Available from: http://www.cadth.ca /media/pdf/compus_Current_Practice _Report_Vol-3-Issue-5.pdf Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 5. Gomes T, Juurlink DN, Shah BR, Paterson JM, Mamdani MM. Blood glucose test strip use: patterns, costs and potential cost reduction associated with reduced testing. Toronto (ON): Institute for Clinical Evaluative Sciences; 2009 Dec [cited 2010 Aug 24]. 21 p. Available from: http://www .ices.on.ca/file/Blood%20Glucose%20Test %20Strip_Dec2009.pdf 12. Blood glucose self-monitoring: benefit is not proven for non-insulin-dependent patients with type 2 diabetes [press release on Internet]. Cologne (DE): Institute for Quality and Efficiency in Health Care; 2009 Dec 14 [updated 2010 Apr 30; cited 2010 Aug 24]. Available from: http: / / w w w. i q w i g . d e / b l o o d - g l u c o s e - s e l f -monitoring-benefit-is-not.997.en.html 6. Farmer A, Wade A, Goyder E, Yudkin P, French D, Craven A, et al. Impact of self monitoring of blood glucose in the management of patients with non-insulin treated diabetes: open parallel group randomised trial. BMJ [Internet]. 2007 Jul 21 [cited 2010 Aug 24];335(7611):132. Available from: http://www.bmj.com/cgi /reprint/335/7611/132 doi:10.1136/bmj.39247 .447431.BE. 7. Boutati EI, Raptis SA. Self-monitoring of blood glucose as part of the integral care of type 2 diabetes. Diabetes Care. 2009;32(Suppl2):S205-10. 8. Cameron C, Coyle D, Ur E, Klarenbach S. Cost-effectiveness of self-monitoring of blood glucose in patients with type 2 diabetes mellitus managed without insulin. CMAJ [Internet]. 2010 Jan 12;182(1):28-34. Epub 2009 Dec 21 [cited 2010 Aug 24]. Available from: http://www.cmaj.ca/cgi/rapidpdf/cmaj .090765v1 doi:10.1503/cmaj.090765. 9. Self Monitoring of Blood Glucose: An Essential Component of Diabetes Management? Preconference panel presentation at the International Diabetes Federation’s 20th World Diabetes Congress; 2009 Oct 18-22; Palais des Congrès de Montréal, Montréal (QC). 10. International Diabetes Federation. Selfmonitoring of blood glucose in non-insulin treated type 2 diabetes. Brussels (BE): International Diabetes Federation; 2009. 44 p. Available from: http://www.idf.org /webdata/docs/SMBG_EN2.pdf 11. Dunbar MJ. Self-Monitoring Blood Glucose Workshop II: development and dissemination of the DCPNS decision tool for self-monitoring blood glucose in on-insulin-using type 2 diabetes. Chronic Dis Inj Can. 2011;32(1):59-61. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 58 Self-Monitoring Blood Glucose Workshop II: development and dissemination of the DCPNS decision tool for self-monitoring blood glucose in non-insulin-using type 2 diabetes M. J. Dunbar, MEd This article has been peer reviewed. Introduction An earlier article described the role of the Nova Scotia Department of Health and Wellness and the work of the Diabetes Care Program of Nova Scotia (DCPNS) and its partners in approaching the controversial topic of self-monitoring of blood glucose (SMBG) in persons with non-insulin-using type 2 diabetes mellitus.1 This preliminary work included the early steps taken to inform, engage and gain consensus on the need for SMBG and frequency of its use in this population and to introduce a draft tool for providers to assist in making decisions around SMBG. Background The question of whether all people with diabetes should self-monitor their blood glucose has received increasing attention in recent years. Individuals and/or the health care system struggle with costs related to testing, the limited evidence in support of testing for some populations Table 1 Responses to consensus questions from the Diabetes Care Program of Nova Scotia (DCPNS) Self-monitoring Blood Glucose (SMBG) Workshop and the utility of SMBG test results in helping individuals manage their disease. In January 2010, the DCPNS invited a multi disciplinary group of diabetes health care professionals to discuss recommendations regarding SMBG for non-insulin-treated type 2 diabetes, specifically its use, frequency and application in Nova Scotia. The proceedings of that workshop were presented in an earlier article.1 Here we discuss the follow-up work, including the refinement and dissemination of the DCPNS Non-Insulin Using Type 2 Diabetes: Decision Tool for Self-Monitoring of Blood Glucose*, and demonstrate the value and the partnerships necessary to support change and promote consistency in approach across provider groups and practice settings. 1.Do all people with non-insulin-using type 2 DM need to test their blood glucose? Post-workshop feedback • 87% – no • 13% – yes, but not routinely 2.Should testing frequency be reduced in non-insulin-using type 2 DM? • 100% – yes, purposefully, on a case-by-case basis 3.For education (self-management purposes), should all people test at diagnosis? • 33% – no • 40% – yes • 27% – should be an option based on individual interest and willingness, blood glucose values, and planned use of results 4. Is a maximum allowance for strips feasible in the non-insulin-using type 2 DM population? • 7% – no • 93% – yes, provided additional qualifiers are considered such as during times of illness 5.Initial self-management education, if appropriate, should focus on staggered, limited SMBG for a specified period of time. Provide your views (what would this look like—how many for how long). • No consensus, responses included ° Not possible to standardize ° 1–2 weeks with SMBG (at variable times and frequencies within) ° 1–4 months Following the January 2010 SMBG Workshop, participants responded to a series of consensus questions. This activity highlighted the power of evidence and thoughtful dialogue in coming to consensus on broad issues and the much more difficult task of reaching agreement on standardized approaches (specifics) due to individual patient and provider differences. The refinement of the decision tool (considerations, examples for testing) and the example of supporting cases (ranging from simple to more complex) is as a result of this feedback (see Table 1). Abbreviations: DM, diabetes mellitus; SMBG, self-monitoring blood glucose. * The Decision Tool is available in Appendix A (online only) from: http://www.phac-aspc.gc.ca/publicat/cdic-mcbc/32-1/ar-09-eng.php#ar0907. Author references: Diabetes Care Program of Nova Scotia, Halifax, Nova Scotia, Canada Correspondence: Margaret (Peggy) J. Dunbar, Suite 548 Bethune Building, 1276 South Park Street, Halifax, Nova Scotia B3H 2Y9; Tel.: (902) 473-3209; Fax: (902) 473-3911; Email: [email protected] 59 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada A “Needs and Wants” exercise was also used to ask participants, “What do you need to help make the changes as discussed [in the SMBG Workshop] a reality in your practice setting?” The participants were to consider this question in the context of each of three categories: individuals with diabetes, health care providers, and other organizations and agencies. These responses were used to support and plan a Nova Scotia–centred approach to SMBG that included partnerships, interdisciplinary sessions, newsletter articles, presentations to key stakeholder groups, the development of educational videos to support self-paced provider learning and patient handouts (see Table 2). Non-Insulin Using Type 2 Diabetes: Decision Tool for Self-Monitoring of Blood Glucose After many iterations and valued feedback from working group members and many others, the DCPNS finalized the one-page decision tool. The intent of the Non-Insulin Using Type 2 Diabetes: Decision Tool for Self-Monitoring of Blood Glucose is to address the need for a more consistent approach to the prescribing and practice of SMBG among and between different health care provider groups (physicians, pharmacists, diabetes educators and others). This colour-coded tool guides and focuses group discussion and individual decisions on issues of greatest concern when considering SMBG†. Four key areas of consideration include: • safety (e.g. risk of hyper- or hypoglycemia); • appropriate and timely action by health care providers based on results of SMBG; • individual’s knowledge, skills and willingness to test and record as well as ability to interpret and act on SMBG results; and • self-management education. The decision tool reinforces critical concepts, prompts yes/no responses to key questions, ensures consideration is given to additional issues that may impact the decision to self-monitor (including age, frailty, cognition Table 2 Responses to Needs and Wants exercise from the Diabetes Care Program of Nova Scotia (DCPNS) Self-Monitoring Blood Glucose (SMBG) Workshop Individuals with diabetes • Education about why and when to test, including rationale and recommendations • Point-of-sale handouts with consistent messaging about when and for how long to test • For those newly diagnosed with diabetes mellitus, emphasize other aspects of self-management such as diet and exercise • A multi-dimensional campaign for promotion through major stakeholders – CDA, Diabetes Centres, pharmacies, physician offices, etc. Health care providers • • • • • • Consistent guidelines with clear recommendations on when and how to test An edited, improved decision tool Inter-professional education through variety of media, including academic detailing Handout for patients explaining the reason for the change in SMBG practice Information on prevention – how to approach, encourage and support necessary changes Policies and education for variety of diabetes care providers (e.g. VON, long-term care managers) and health care educators (e.g. community college and university programs) • Articles in DCPNS newsletter, Pharmacare newsletter, etc. Other agencies and organizations (e.g. CDA, DHW, Medavie BlueCross, etc.) • New evidence-based guidelines – CDA should play key role in supporting/disseminating message about change in SMBG through its patient and provider publications, website, etc. • Collaboration between agencies • Mailings to clients who use the provincial government Pharmacare services, private insurers such as Medavie Blue Cross, etc. • Distribute “best practice” information to relevant agencies • Education about SMBG and how to access programs and services Abbreviations: CDA, Canadian Diabetes Association; DCPNS, Diabetes Care Program of Nova Scotia; DHW, Department of Health and Wellness; VON, Victorian Order of Nurses. and finances), provides examples of highand low-intensity testing, and reinforces the need for time-limited testing in those who do test. Mindful of the need for information and education through a variety of media, two short educational videos support the dissemination and uptake of the decision tool. Video 1 (SMBG Decision Tool for Health Care Providers) provides the rationale for the decision tool in light of the evidence and local considerations. Key opinion leaders provide their insights on SMBG in the non-insulin-using type 2 diabetes mellitus population, the rationale for the change in practice, the opportunities that this change creates for both patients and providers, and the value of the decision tool to reduce subjectivity and promote a more thoughtful approach to SMBG. Video 2 (Use of the SMBG Decision Tool and Case Studies) introduces the tool and illustrates how to use it. The video highlights the features of the tool, works through a sample case, summarizes principles and caveats to guide future application, and presents three additional case studies (from those newly diagnosed to those with long-standing diabetes) for providers to work through on their own.‡ Although the official launch was to be in September 2010, the tool (without the videos) was first introduced to physicians, pharmacists and diabetes educators in May 2010 through academic detailing sessions conducted by the Office of Continuing Medical Education at Dalhousie University. The tool and the videos became the focus of inter-professional workshops held across Nova Scotia as of February 2011. These community-based sessions continue to be offered free of charge to physicians, diabetes educators and community pharmacists as well as interested inpatient, ambulatory care and community health care professionals. Supported by a local clinical expert, representatives from Dalhousie University’s Departments of Continuing Medical and Pharmacy Education, Capital Health’s Drug The Decision Tool is available in Appendix A (online only) from: http://www.phac-aspc.gc.ca/publicat/cdic-mcbc/32-1/ar-09-eng.php#ar0907. † ‡ The decision tool and videos are available from http://www.diabetescareprogram.ns.ca. Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 60 Evaluation Unit and the DCPNS lead the sessions. Each 90-minute session includes role-playing, overview of the evidence (with a focus on the local context), use of the SMBG Video 2 to introduce the decision tool and its various features followed by case-based, small group work led by the clinical expert. Next steps Opportunities to promote the tool and the need for consistency in approaches to SMBG continue to present themselves in the form of abstract submissions, conference presentations, speaking engagements, and sharing across provinces and agencies that have an interest in this topic. An evaluation plan is currently under develop ment; it will include monitoring prescribing practices though the Nova Scotia Department of Health and Wellness Pharmacare Program and a review of diabetes educator practices related to use of the tool and approach to counselling. Currently, DCPNS is leading the deve lopment of a parallel decision tool aimed at individuals with diabetes. This tool will explain why the recommended SMBG practices have changed and will include a simple self-test to assist individuals in determining if they need SMBG. For those needing to test, simple guidelines will explain when and how often to do so. Acknowledgements The Diabetes Care Program of Nova Scotia would like to thank all the participants of the Self-Monitoring Blood Glucose Workshop and the staff of the DCPNS for helping to guide this important work, as well as the observers for their encouragement and support. We would also like to acknowledge the commitment of Dalhousie University Departments of Continuing Medical and Pharmacy Education as well as the Drug Evaluation Unit, Capital Health, for their collaborative approach and valued partnerships in the planning and delivery of interdisciplinary education sessions across Nova Scotia. A special thanks to Pam Talbot for her contributions to the preparation of this manuscript. References 1. Dunbar MJ. Self-Monitoring Blood Glucose Workshop I: promoting meaningful dialogue and action at the provincial level. Chronic Dis Inj Can. 2011;32(1):55-58. This continued work will benefit from the insight of many partners who have provided support, encouragement and perspective. SMBG is not just a diabetes educator issue; it affects all providers across multiple settings who interact with people who have diabetes as well as individuals living with diabetes and their family members. A measured approach to SMBG will benefit individuals with diabetes: less testing means happier fingers and more effective use of personal health care dollars without compromising care or health outcomes. The health system will also benefit from more appropriate use of SMBG by reducing the burden of unnecessary and wasteful testing. 61 Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada Book review Nutraceuticals, Glycemic Health & Type 2 Diabetes N.K. Bonsu, MSc Editors: Vijai K. Pasupuleti and James W. Anderson Publisher: Wiley-Blackwell Publishing Publication date: August 2008 Number of pages: 512 Format: Hardcover Price: $275.99 CDN ISBN: 9780813829333 Diabetes is one of the fastest growing chronic diseases globally and is the fourth or fifth leading cause of mortality in many developing and newly industrialized countries. Most methods of preventing or managing this insidious disease involve the use of drugs. However, individuals diagnosed with diabetes are increasingly searching for more natural products to prevent and manage this disease. As a result, the editors decided to examine the effect that nutraceuticals have on the glycemic health of those individuals diagnosed with type 2 diabetes and present the latest nutraceutical research in this book. They used a diverse assortment of contributors, from academia, industry and government, to compile the various chapters that make up this book. Similarly, the intended audience are researchers in academia and industry, epidemiologists, biostatisticians and health care workers, though consumers of nutraceuticals from the general public who would like a detailed scientific analysis of the research could also make use of it. The book is divided into three sections. The first is composed of a single chapter that provides a brief overview of the various causes of diabetes as well as its prevention and management. This chapter also shows the linkage between nutra ceuticals and diabetes prevention and management. The second section consists of five chapters and deals with glycemic health and type 2 diabetes. The first chapter in this section begins by providing an overview of the epidemiology of type 2 diabetes. The second chapter describes various international studies that have linked lifestyle changes in diet and exercise with prevention of type 2 diabetes as well as various pharmacological approaches. The final three chapters in this section deal with the causes of hyperglycemia and the resulting health implications and introduce the reader to the controversial aspects of diet and the glycemic index of foods. The final section in the book is by far the most comprehensive. It provides a detailed analysis of various functional foods and nutraceuticals, including ones from among traditional Chinese medicine and Indian and Mexican herbs and plants, that have proven health benefits to those diagnosed with type 2 diabetes. Some of the nutraceuticals discussed in separate chapters include dietary fibre, cinnamon, soybeans and ginseng as well as minerals and natural resistant starches. Those that have not been fully tested Author references: University of Regina, Regina, Saskatchewan, Canada Vol 32, No 1, December 2011 – Chronic Diseases and Injuries in Canada 62 have shown promising results, but more research will be needed on those specific nutraceuticals. This final section ends with a short chapter that examines future trends and directions in this area. Each of the topics is extensively researched and documented by the respective contributors. The chapters are very well written, all of the references are fairly current and relevant, and many figures and tables complement the text. This is a great book about the different nutritional interventions that can be used to combat type 2 diabetes. It is also timely, given the need for more effective means to control the incidence and prevalence of this disease and the increase in popularity of natural remedies. The benefits of many nutraceuticals are not well known. This detailed summary of the available research makes it easier to access the pertinent information, making this book a suitable addition to the literature. Researchers, epidemiologists, biostatisticians and health care workers will find this compilation to be a useful reference tool, as would senior students who are familiarizing themselves with the epidemiology of diabetes and different prevention and treatment methods. CDIC: Information for authors Chronic Diseases and Injuries in Canada (CDIC) is a Submitting Manuscripts References: In Vancouver style (consult a recent quarterly scientific journal focusing on the prevention CDIC issue for examples); numbered in superscript and control of non‑communicable diseases and Submit manuscripts to the Managing Editor, injuries in Canada. Its feature articles are peer Chronic Diseases and Injuries in Canada, Public up to six authors (first six and et al. if more); reviewed. The content of articles may include Health Agency of Canada, 785 Carling Avenue, without research Address Locator 6807B, Ottawa, Ontario K1A 0K9, feature used in word processing; any unpublished email: [email protected] observations/data or personal communications from such public/community fields health, as epidemiology, biostatistics, the behavioural sciences, and health services or in the order cited in text, tables and figures; listing any automatic reference numbering used (discouraged) to be cited in the text in economics. CDIC endeavours to foster communi‑ Since CDIC adheres in general (section on parentheses (authors responsible for obtaining cation on chronic diseases and injuries among illustrations not applicable) to the “Uniform written permission); authors are responsible for public health practitioners, epidemiologists and Requirements verifying accuracy of references. researchers, health policy planners and health to Biomedical Journals” as approved by the educators. Submissions are selected based on International Journal Tables and Figures: Send vector graphics only. scientific quality, public health relevance, clarity, Editors, authors should refer to this document for Each on a separate page and in electronic file(s) conciseness and technical accuracy. Although complete details before submitting a manuscript separate from the text (not imported into the text CDIC is a publication of the Public Health Agency to CDIC (see <www.icmje.org>). body); as self‑explanatory and succinct as possible; of Canada, contributions are welcomed from both the public and private sectors. Authors retain responsibility for the contents of their papers, and for Manuscripts Committee of Submitted Medical Checklist for Submitting Manuscripts opinions expressed are not necessarily those of the not too numerous; numbered in the order that they are mentioned in the text; explanatory material for tables in footnotes, identified by lower‑case superscript letters in alphabetical order; figures CDIC editorial committee nor of the Public Health Cover letter: Signed by all authors, stating that limited to graphs or flow charts/templates (no Agency of Canada. all have seen and approved the final manuscript photographs), with software used specified and and have met the authorship criteria including a titles/footnotes on a separate page. Article Types full statement regarding any prior or duplicate publication or submission for publication. Peer‑reviewed Feature Article: Maximum 4000 Number of copies: If submitting by mail, one complete copy, including tables and figures; one words for main text body (excluding abstract, First title page: Concise title; full names of all copy of any related supplementary material, and a tables, figures, references) in the form of original authors and institutional affiliations; name, postal copy of the manuscript on diskette. If submitting research, surveillance reports, meta‑analyses or and email addresses, telephone and fax numbers by email to cdic‑[email protected]‑aspc.gc.ca, please methodological papers. for corresponding author; separate word counts for fax or mail the covering letter to the address on the abstract and text. inside front cover. Status Report: Describe ongoing national programs, studies or information systems bearing on Canadian Second title page: Title only; start page numbering public health (maximum 3000 words). Abstract not here as page 1. required. Abstract: Unstructured (one paragraph, no Summarize headings), maximum 175 words (100 for short relating to reports); include 3-8 keywords (preferably from national public health (maximum 1200 words). the Medical Subject Headings [MeSH] of Index Abstract not required. Medicus). Cross‑Canada Forum: For authors to present or Text: Double‑spaced, 1 inch (25 mm) margins, exchange information and opinions on regional or 12 point font size. Workshop/Conference significant, recently Report: held events national surveillance findings, programs under development or public health policy initiatives Acknowledgements: Include disclosure of financial (maximum 3000 words). Abstract not required. and material support in acknowledgements; if anyone is credited in acknowledgements with Letter to the Editor: Comments on articles substantive scientific contributions, authors should recently published in CDIC will be considered for state in the cover letter that they have obtained publication (maximum 500 words). Abstract not written permission. required. Book/Software Review: Usually solicited by the editors (500-1300 words), but requests to review are welcomed. Abstract not required.
* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project