provides a systematic and transparent framework for clarifying
questions, determining the outcomes of interest, summarizing
the evidence that addresses a question, and moving from the
evidence to a recommendation or decision. The system is designed
for reviews and guidelines that examine alternative management
strategies or interventions, which may include no intervention
or current best management. GRADE may be applied to a broad
scope of clinical questions, including diagnosis, screening,
prevention and therapy.
The Guidelines Committee (GC) of NCS chose GRADE
methodology over alternate systems because of its
simplicity, flexibility and adaptability (Figure 1). In GRADE,
recommendations have two components: a two-level
representation of the strength of recommendation (strong
or weak) and a four-level representation of the quality
of the evidence (high, moderate, low and very low)7. A
recommendation is a strong recommendation to do (or not
do) something, where the benefits clearly outweigh the risks
(or vice versa) for nearly all patients. The quality of evidence in
GRADE reflects confidence in the estimates of benefits, harms
and burdens. Prior grading systems had many disadvantages,
including the lack of separation between the quality of evidence
and strength of recommendations, the lack of transparency about
judgments, and the lack of explicit acknowledgment of values
and preferences underlying the recommendations. In addition,
the existence of many, often scientifically outdated, grading
systems has created confusion among guideline developers and
end users. The GRADE approach was developed to overcome the
shortcomings of previous grading systems. Advantages of GRADE
over other grading systems include:7
• Developed by a widely representative group of international
guideline developers
• Clear separation between judging confidence in the effect
estimates and strength of recommendations
• Explicit evaluation of the importance of outcomes of
alternative management strategies
• Explicit, comprehensive criteria for downgrading and
upgrading quality of evidence ratings
• Transparent process of moving from evidence to
recommendations
• Explicit acknowledgment of values and preferences
• Clear, pragmatic interpretation of strong versus weak
recommendations for clinicians, patients, and policy makers
• Useful for systematic reviews and health technology
assessments, as well as guidelines.
More than 100 organizations from 19 countries around the world
have endorsed or are currently using the GRADE process.
The Guideline Development Process at NCS
Development of CPGs by the NCS takes approximately two years
from identification of the topic to publication and dissemination.
Our process is outlined in Table 1 and detailed below:
1. Perform a needs assessment: The critical first step in CPG
development is to determine an existing deficiency in clinical
practice. NCS performs a needs assessment by querying its
membership annually. At the beginning of each calendar year,
NCS sends out a member survey to identify possible guideline
topics. Additionally, other medical societies often invite us to
collaborate on CPG development on topics to meet the needs of
the broader medical community. An optimal CPG topic addresses
a disease that is common or a population that is large, where
practice changes would be most impactful. However, the topic
must be feasible. The IOM has identified the following prioritysetting
criteria for guideline development:
• Disease burden – extent of morbidity and mortality imposed
by a condition
• Controversy – degree of controversy or uncertainty about a
particular topic
• Cost – economic cost associated with a particular condition
• New evidence – new data exists with the potential to change
prior conclusions
• Potential impact – potential to improve decision-making for
patient or provider
• Public or provider interest – stakeholders want to inform
decision-making
• Variations in care – reduce variations in prevention,
diagnosis, or treatment
2. Determine suitable topics: The results of the member surveys and
external society requests are collated and tallied. The Guidelines
Committee (GC) discusses the possible options, briefly reviews
the extant literature and votes to determine the three to five best
topics. These topics are ranked and presented to the Executive
Committee, which makes the ultimate decision about the next
guideline topic at the annual meeting in March.
3. Formation of a balanced writing group: After determination of
the topic, the GC chooses topic co-chairs. Co-chairs are chosen
largely for their expertise and knowledge of the subject matter.
Prior guidelines experience is preferred for at least one chair.
With guidance from the GC, the topic co-chairs assemble a
multidisciplinary and well-balanced team of experts in the
relevant clinical topic. The group should be balanced with regard
to profession (e.g., MD, RN, PharmD), seniority, sex/gender,
race, guidelines experience, nationality and membership status.
Determination of whether patient groups or advocates should be
involved in the process is made at this stage. If partnership with
external societies is enviable, the GC will reach out to establish
a memorandum of understanding and to determine whether a
formal designee should be included on the writing committee.
4. Train in the GRADE process: After the writing committee has
been assembled, the next step is to train members in GRADE
methodology. Every writing group is assigned a GRADE Coach,
or methodologist, who has attended national GRADE training
conferences. The GRADE Coach educates the writing committee
on GRADE methodology and formulation of appropriate
questions. These questions are framed using the PICO (P-patient,
I-intervention, C-comparators, O-outcomes) criteria. Questions
are limited to approximately 10-15.
5. Consult a librarian: Once the PICO questions are formulated,
a librarian is consulted to assist in an unbiased and systematic
review of the literature. It has become more common for librarians
to assist with systematic reviews and CPGs as ‘‘best practice’’
standards evolve for guideline development.8
6. Evaluate the extant literature: A systematic review and metaanalysis
of relevant literature is then performed for each specific
PICO question. The evidence is then rated using dimensions
defined by GRADE: risk of bias, inconsistency, imprecision,
indirectness, publication bias, large effect or dose effect of the
intervention, and plausible residual confounding. We currently
employ a software program designed to standardize and
facilitate this review process. The web-based application uploads
5