Top Tips for Turning Information Into Insights

Top Tips for Turning Information Into Insights
Googling The Greys
Tips for Searching Beyond
Health Databases
Turning Information into Insights
Sarah Bonato
Reference Librarian
Centre for Addiction and Mental Health
Ontario Library Association
January 23, 2013
Topics to Be Covered
Definition of grey literature
Benefits of adding a grey literature search
Examples of traditional grey literature sources
Example of untraditional grey literature sources
How grey literature can provide added value to reference searches
Googling for Grey Lit versus using standardized checklists
Tips & Tricks for using Google for grey literature searching
Personalized searching--drawbacks
Bypassing location in Google
Searching Google, Google UK, Google Australia, Google US government search
Alternative search engines to Google
Sharing, appraising and annotating grey literature search results
Definition of Grey Literature
"Information produced on all levels of government, academia,
business and industry in electronic and print formats not
controlled by commercial publishing i.e. where publishing is
not the primary activity of the producing body."
(ICGL Luxembourg definition, 1997 - Expanded in New
York, 2004).
Definition of Grey Literature, Con’t
”The information and resources that do not categorically fall
into what is available via standard traditional or commercial
publishing channels.”
International Journal on Grey Literature
Why Add Grey Literature to Search
Searching for unpublished studies or data reduces level of possible bias
– Exclusion of grey literature from meta analyses can lead to exaggerated
estimates of intervention effectiveness
» Laura McAuley, Ba’ Pham, Peter Tugwell, David Moher.
“Does the inclusion of grey literature influence estimates
of intervention effectiveness reported in meta-analyses?”
Lancet 2000; 356:1228–31
Grey literature documents provide detailed overviews on specific
Excellent for starting a search on subjects that are not concrete enough for
databases searches
Why Add Grey Literature to Search
Results? (Con’t)
• Grey literature adds a valuable global perspective to search
• Grey literature documents provide detailed overviews on
specific treatment populations
• Grey literature may be the only source of specific local
• Grey literature may provide more concrete examples of
implementing policies or procedures
• Grey literature may be faster and easier for your patron to
quickly read through
Why Add Grey Literature to Search
Results? (Con’t)
• Most Grey Literature is available for free
• Can be published and disseminated quickly
• Can locate information intended for specific
– Nurses
– Occupational therapists
– Mental health clinicians
So that PubMed Search Isn’t
• Journal articles are an excellent information source,
– May not provide a broad overview, instead focusing on the
outcome of specific interventions
– Very focused on clinical trails
– May not provide summary and background history on
– May results in many irrelevant hits
– May miss key policy sources
– Lacking in examples of practical execution
– Grey literature essential for searches on best practices
Traditional Sources of Grey
• Governmental reports
– publications from the local, national and
international sources
• NGO publications
• Dissertations
• Think Tank publications
• Guidelines and position papers from professional
• Publications from national and international health
Key Grey Lit Source-Government
• Government documents can be from a
municipal, state/provincial or national source
• Excellent source of information, since data is
often collected at the local and provincial/state
• Essential for health systems research
Examples of Grey Lit from
Governmental Sources
• National Examples:
– Public Health Agency of Canada: Findings from Enhanced
Surveillance of Canadian Street Youth 1999-2003
– National Health and Medical Research Council
(Australia): Clinical Practice Guidelines- Familial Aspects
of Cancer: A guide to clinical practice
– Substance Abuse and Mental Health Service
Administration (US) : Preventing Suicide A Toolkit for
High Schools
Examples of Grey Lit from
Governmental Sources Con’t
• Provincial/State Examples
– Ontario Ministry of Health and Long Term Care: A
Program Framework for Mental Health Diversion/Court
Support Services
– Vermont Department of Health: Recommendations for
Management of Diabetes for Children in School
Examples of Grey Lit from
Governmental Sources Con’t
• Municipal Examples:
– Toronto Public Health: Health Status News—Toronto
Breastfeeding Rates
– New York City Department of Health and Hygiene:
Healthy Homes: Keeping Housing Healthy and Safe for
Types of Untraditional Grey
Literature Publications
Health Consumer Organizations
PowerPoint Presentations
Untraditional sources may provide excellent information
Come out, come out
wherever you are!
Many national and international organizations publish grey literature , but
can be difficult to identify
Main sources for grey literature
– The New York Academy of Medicine Library’s Grey Literature Report
Web site checklists for grey literature
– CADTH’s Grey Matters: a practical search tool for evidence-based
Web sites or databases tracking specific grey literature sources can be
retired at any time
– US Department of Energy's Office of Scientific and Technical
Information’s GrayLit Network—a Science Portal of Technical Reports
Googling for Grey Literature
• Excellent for publications not indexed by databases
• Speeds up searching
• Great for identifying publications from organizations that may
be unfamiliar
• No need to rely on checklists that may not be outdated or only
focus on larger organizations
Google Personalized Searching, For
• Launched in 2009
• Customized results according to past search history linked to
your browser using cookies and location by IP address
• Google has been assigned a patent for personalized network
• Personalized search history can be turned off, but location still
will influence hits
Google Personalized Searching
• Search results are influenced by search histories
• Browser version—Google assumes that anything less than the
latest version means a less sophisticated searcher
The Filter Bubble: What the Internet is Hiding from You, by
Eli Pariser
• Personalized hits criticized for poorer quality information,
lacking in diverse results
• Geographic location influences search results immensely
Location, Location, Location and
Google Searching
But in order to find diverse information, you
have to look beyond the Google you always
see and use…
Incorporating Location into a
Google Grey Literature Search
• Broadening a Google Search to include hits from diverse
locations can be accomplished by:
– Using country specific Google search engines and limiting
results to that country
– Limiting by a country domain in Google Advance Search
– Using Google Advance Search and limiting to region
Using Country Specific Google
Search Engines
• Easiest way to find diverse grey lit search results
• Can focus on any country
• Look beyond our borders—Ireland and New Zealand
are excellent sources of grey lit
Example guideline search subject—
Recommended Sodium Intake
Recommended sodium intake guidelines vary
Using country specific Google searches can be designed to retrieve geographically
unique content
The US, Canada, United Kingdom, Australia, and New Zealand do not have the
same recommended sodium intake values:
– Canada 1,000 -1,500 mg of sodium per day
– Australia 1,600 mg of sodium per day
– New Zealand 1,600 to 2300 mg of sodium per day
– United Kingdom 1,600 sodium mg per day
– US guidelines-- intake is recommended at no more than 2,300 mg
of sodium per day. Those with hypertension, African-Americans,
and middle-aged and older adults should limit intake to 1,500 mg
of sodium per day
Demo Searches
• Google
• Google UK
• Google Australia
Example Searches
A grey literature search on sodium intake guidelines can include hits from
Google Canada, Google UK, Google Ireland, Google Australia, Google
New Zealand
Purposefully retrieving content from specific geographical regions adds
diversity and wide range to results
– Also you are locating content that patron may not have been able to
Varying your search terms is a good way to find hits for different reading
levels and populations.
– Providing different format content is added value to search results
Search in Just Plain Google
• Search terms:
sodium intake guidelines
Google UK
• Use Google to find Google UK
• Conduct a Search
• Under search tools, limit the search to pages
from the UK
Benefits of Searching Google UK
• National Health Service produces large
quantity and superior quality of grey lit
• NHS regions produce unique content
• Other UK organizations also have quality grey
literature content
Google UK Search
Google UK Search
Without limiting to UK pages
Limiting to Pages from the UK
• Search terms:
sodium intake guidelines
limited to UK results
UK Google Search Results
Searching Google Australia
• Google Australia an excellent source of grey
literature results
• Shared similarities results in relevancy
– Universal Health Care
– Immigration
– Aboriginal Populations
Search in Google Australia
Google Australia Search
Without Limiting to Australian Pages
Search in Google Australia
Search terms:
sodium intake guidelines
limited to Australian results
Searching in Google Advance Search
With Region Limit
• Alternative to using country specific Google
Search engines
• Search results are very similar, but ranking of
sites are different
• Google Advance Search allows for more
search filters to be applied on one page
Sample Google Advanced Search
• Example search subject
– Trauma Informed Care
• Searching for term as a phrase
– Search limited to pages from Canada
Google Advanced Search
Search Results
Searching in Google Advance Search
Using Domain Limits
• Domain limits are a quality filter to retrieve more
relevant search results
• Using domain limits may be the only way to retrieve
content that would be lost in a Google search
• Best bests domain limits:
– org
– edu
– int
– gov
Key Grey Lit Searching Tip—
Limiting to gov
• Much quality content is available from US sources
• National, state and local US governments produce
quality grey lit
• Locating can be difficult, due to vast amounts of
available information
• Just limiting to US region does not retrieve quality
hits, more filters are needed
• Must exclude PubMed hits
– Google search command is -pubmed
Search in Google limited to
domain gov
• Search terms: sodium intake guidelines
• Limited to the domain gov and –pubmed
Search Results
Using Domain
• Domain org an excellent filter to use for grey
literature searches
• Can be used in Google Advanced Search or by
• Best combined with other limits to increase
retrieval of primarily grey literature hits
Sample Search
• Sample search topic:
– student support and mental illness
• Search limits– org
– Exclude PubMed hits
– Limit to file type PDF
Google Search
Google Search Results
“Be obscure clearly”
E.B. White
• When Googling for grey literature remember:
• Different terms and vocabulary are used to capture like
• Not varying your search terms will result in fewer and
less diverse hits
– Examples: university students, college students, youth,
adolescent, aboriginals, Native Americans, First Nations.
And Sometimes Google isn’t always
the greatest
• Google Scholar does not include most quality grey lit
• Grey lit searching for systematic reviews should
include specific sites, not just a google search
– IE
• Using or creating customized Google Search Engines
– Good for grey lit searches, only when you know
all of the sources that should be searched
Ways to Filter Search Results
Search for specific domain endings—i.e. org, edu
Exclude domain endings—ie –pubmed in a US search
Vary between British and American spelling or phrases—i.e. use search terms
ministry of health or youth or aboriginal
Select or add specific file types
Select year range
Reading level
Exclude hits from specific journals
Regions—limit results to a specific region
Allintext to search for documents with specific words in the text
~ to find similar terms. Beware—this is focused very much on terms and phrases
commonly used in the US
Use link: to find sites that link to a desired URL or related: to find web pages that
are similar to the URL. A great way to find organizations that you are not familiar
Alternatives to Google
(YES Virginia, there ARE alternatives!)
• DuckDuckGo
Search results are not personalized, so different
users will receive the same results when an identical strategy is
• Startpage
Offers the web search results directly from
Google, but its privacy settings means that location and
pervious search histories are not recorded
Google Search on
Safer Injection Facilities
DuckDuckGo Search on
Safer Injection Facilities
Startpage Search on
Safer Injection Facilities
Sample of Recent
Grey Literature Searches
• Respectful language use with psychiatric/addiction clients
• Community partnerships
• Best practices for youth with concurrent disorders
• Nutrition & Seniors
• Ultra Rapid Detox for heroin abuse
More Samples of Grey Literature
• Checklist for evaluating housing for fire safety and child safety
– Content too specific to locate in journal literature
• Housing policy and homelessness
– Journal literature focused on specific interventions
• Mentoring and clinical placements
– Journal literature did not provide broad overview
• Smoke free hospital campuses—practical implementation
– journal literature hits were older
Critical Appraisal of Grey
Literature Results
• Traditional steps of critical appraisal can be applied
to grey literature sources
– Easily can identify producing source
– Conflicts of interest
• References, funding, reputation of organizations
– Data justifying conclusions
– Relevancy of publication
Grey Literature
Search Results
• Email with links
• Provide overview with links
– Content
– Title
– Authorship
– Location of relevant information
– Other notations
Sample Email
Sample Email Cont’d
• Grey literature provides valuable content to patron requests
• Goggling for grey literature is a quick way to add diversity and
unique content to search results
• Savvy searching will allow you to bypass Google’s
personalized search results
• In order to find and include diverse search hits, you must take
steps to bypass search results based on search location
“Google assigned patent for personalized network searching” Targeted News Service (2011, Sep 07) pp.
Bates, Mary Ellen. "Super Searcher Strategies & Tips“ Internet Librarian, Monterey, CA. October 2011.
Chan An-Wen. “Out of sight but not out of mind: how to search for unpublished clinical trial evidence”
BMJ (2011, Jan 3) d8013.
Lehman Richard and Loder Elizabeth. “Missing Clinical Trial Data: A threat to the integrity of evidenced
based medicine”. BMJ ( 2012; Jan 3) d8158
Fox, Vanessa. “The Anatomy of Personalized Google Results” Information Today (2007, Nov) pp 24.
Mullally, U. “Why the web is telling us what it thinks we want to hear” Irish Times (2011, Jun 29) pp. 15.
Pariser, Eli. “The Filter Bubble—What the Internet is Hiding From You” New York : Penguin Press,
Winston, Stephanie Dalhousie. “Don’t Be Evil—Uncovering the Implications of Google Search” Journal
of Interdisciplinary Management (2011 Mar 31) pp 7-12.
Sodium intake guidelines retrieved on 5/31/2012 from:
Health Canada ( )
Nutrition Australia (
New Zealand Ministry of Health (
National Health Services ( )
USDA Center for Nutrition Policy and Promotion (
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF