GRADE-beoordeling van de in de review geïncludeerde artikelen

Toelichting op de methodologie

GRADE is een internationaal erkend instrument voor het beoordelen van de kwaliteit van het wetenschappelijk bewijs en het opstellen van de richtlijnenaanbevelingen. De nader toelichting van de methode van GRADE-beoordeling is beschikbaar online op http://gdt.guidelinedevelopment.org/app/handbook/handbook.html.

 

Bij het beoordelen van de kwaliteit van het wetenschappelijk bewijs worden diverse factoren gebruikt (zie onderstaande tabel):

Factors that can reduce the quality of the evidence

Factor

Consequence

Limitations in study design or execution (risk of bias)

↓ 1 or 2 levels

Inconsistency of results

↓ 1 or 2 levels

Indirectness of evidence

↓ 1 or 2 levels

Imprecision

↓ 1 or 2 levels

Publication bias

↓ 1 or 2 levels

 

Factors that can increase the quality of the evidence

Factor

Consequence

Large magnitude of effect

 1 or 2 levels

All plausible confounding would reduce the demonstrated effect or increase the effect if no effect was observed

 1 level

Dose-response gradient

 1 level

 

De GRADE-methode is nog niet optimaal voor de beoordeling van studies van diagnostische accuratesse, met name met betrekking tot ‘inconsistency’and ‘imprecision’van het bewijs (Hoofdstuk 7.5.2.3 van het GRADE Handbook). ‘Inconsistency’betreft de niet verklaarbare, statistisch significante heterogeniteit van de uitkomstmaten van diagnostische accuratesse studies (sensitiviteit, specificiteit, voorspellende waardes, etc.). De methoden en criteria van de beoordeling van ‘Imprecision’zijn minder goed opgeschreven voor diagnostische onderzoeksvragen in vergelijking met therapeutische interventies.

Bij het opstellen van deze richtlijn was het vaak niet mogelijk om ‘inconsistency’van het bewijs te beoordelen, omdat er weinig artikelen in de literatuuranalyse zijn geïncludeerd. In deze richtlijn werden meermaals gegevens uit één artikel per uitgangsvraag gebruikt. In geval van meer dan één artikel werd er waar mogelijk gebruik gemaakt van de range van specificiteit en sensitiviteit, waarbij de hoogste vermelde waardes van deze uitkomsten zijn meegenomen in de beoordeling.

 

De afwaardering van de GRADE-score bij de beoordeling van de bewijskracht van geïncludeerde studies werd gedaan op basis van de volgende criteria:

  • de studie opzet en de manier van uitvoeren van de studie (case-controle studies zijn met
    1 punt, van ‘high’naar ‘medium’afgewaardeerd);
  • het risico op bias (aan de hand van het QUADAS-2 instrument; de studiekwaliteit is met
    1 punt afgewaardeerd, als er sprake was van mogelijke bias);
  • ‘indirectness’; dit betreft verschillen in de referentietest (anamnese of orale provocatie), verschillen tussen protocollen voor orale provocaties tussen de artikelen, inclusie van gezonde personen bij het berekenen van de sensitiviteit en specificiteit (bias kan ontstaan omdat de gezonde vrijwilligers niet behoren tot de populatie waarin de indextest toegepast gaat worden);
  • ‘imprecision’; dit betreft te kleine patiëntengroepen en het gebruik van 1 studie per uitgangsvraag (waardoor de spreiding van de juiste afkapwaarde voor de uitkomsten lastig in te schatten is).

Toelichting op de resultaten

De onderstaande tabellen zijn gemaakt met behulp van de online GRADE-software (https://gradepro.org/). De in de tabellen vermelde prevalentiecijfers zijn gebaseerd op de gegevens uit de literatuur en ervaring van werkgroepleden, omdat de prevalentie voor de studiepopulaties in de geïncludeerde artikelen niet wordt vermeld. Aan de hand van de ingevulde (afgeronde) prevalentiecijfers, de sensitiviteit en de specificiteit wordt de kans op het opsporen van NSAID-overgevoeligheid met de desbetreffende test (per 1000 patiënten) automatisch door de software berekend. Onder ‘Prevalences of NERD in specific populations’zijn drie prevalentiecijfers weergegeven: 3% is de aangenomen prevalentie van NERD bij patiënten met astma in de allergiecentra, 7% en 9% zijn de aangenomen prevalenties van NERD bij astmapatiënten uit de twee recente meta-analyses2,3.

 

Voor de interpretatie van de tabellen kan een voorbeeld van de 15-HETE generation assay worden genomen. De cijfers onder ‘Effect per 1000 patients tested’bij 7% betekenen dat 26% (58 true positives uit 225 positieve resultaten per 1000 testen) van de mensen met een positieve uitslag van deze test inderdaad NERD hebben, terwijl 2% (12 false negatives uit 775 negatieve resultaten) van de mensen met een negatieve uitslag niet zijn opgespoord door de test en ook NERD hebben.

 

Literatuur:

1. The GRADE Handbook. http://gdt.guidelinedevelopment.org/app/handbook/handbook.html. Accessed on August 31, 2015.

2.Rajan JP, Wineinger NE, Stevenson DD, White AA. Prevalence of aspirin-exacerbated respiratory disease among asthmatic patients: A meta-analysis of the literature. J Allergy Clin Immunol. 2015;135:676-81.e1.

3. Morales DR, Guthrie B, Lipworth BJ, Jackson C, Donnan PT, Santiago VH. NSAID-exacerbated respiratory disease: a meta-analysis evaluating prevalence, mean provocative dose of aspirin and increased asthma morbidity. Allergy. 2015;70:828-835.

 

Question: Should 15-HETE generation in peripheral blood leukocytes test be used to diagnose NSAID hypersensitivity in adult patients with NERD? (Kowalski, 2005)

Sensitivity

0.83

Specificity

0.82

 

Prevalences of NERD in specific populations

7%

3%

9%

 

 

Outcome

№ of studies (№ of patients)

Study design

Factors that may decrease quality of evidence

Effect per 1000 patients tested

Test accuracy QoE

Risk of bias

Indirectness

Inconsistency

Imprecision

Publication bias

pre-test probability of 7%

pre-test probability of 3%

pre-test probability of 9%

True positives
(patients with NSAID hypersensitivity)

1 study
95 patients

case-control type accuracy study

very serious 1,2

serious 3

not serious 4

not serious

none

58

25

75

⨁◯◯◯
VERY LOW

False negatives
(patients incorrectly classified as not having NSAID hypersensitivity)

12

5

15

True negatives
(patients without NSAID hypersensitivity)

1 study
95 patients

case-control type accuracy study

very serious 1,2

serious 3

not serious 4

not serious

none

763

795

746

⨁◯◯◯
VERY LOW

False positives
(patients incorrectly classified as having NSAID hypersensitivity)

167

175

164

  1. The 15-HETE test was compared between ASA tolerant and ASA intolerant asthmatics, but is not compared with OPT in all patients. Some patients were diagnosed with bronchial PT and others based on clinical history. The study population differs in the number of patients with nose polyps per comparison arm.
  2. Patient selection and patient flow might have introduced bias.
  3. PICO in the study differs from PICO of this guideline. Indirect comparison between different provocations and index test.
  4. Inconsistency couldn't be assessed.

 

Question: Should basophil activation test (CD63, CD69, CD203c) be used to diagnose NSAID hypersensitivity in patients with NERD? (Celik, 2009)

Sensitivity

0.30

Specificity

0.40

 

Prevalences of NERD in specific populations

7%

3%

9%

 

 

Outcome

№ of studies (№ of patients)

Study design

Factors that may decrease quality of evidence

Effect per 1000 patients tested

Test accuracy QoE

Risk of bias

Indirectness

Inconsistency

Imprecision

Publication bias

pre-test probability of 7%

pre-test probability of 3%

pre-test probability of 9%

True positives
(patients with NSAID hypersensitivity)

1 study
20 patients

case-control type accuracy study

very serious 1

very serious 2

not serious 3

 serious 4

none

21

9

27

⨁◯◯◯
VERY LOW

False negatives
(patients incorrectly classified as not having NSAID hypersensitivity)

49

21

63

True negatives
(patients without NSAID hypersensitivity)

1 study
20 patients

case-control type accuracy study

very serious 1

very serious 2

not serious 3

serious 4

none

372

388

364

⨁◯◯◯
VERY LOW

False positives
(patients incorrectly classified as having NSAID hypersensitivity)

558

582

546

  1. Spectrum bias due to inclusion of healthy controls and case-control design.
  2. Healthy subjects would normally not be subjected to the index test. Healthy subjects comprise the whole control group.
  3. Inconsistensy could not be assessed.
  4. Study population very small and does not include patients with NSAID tolerant asthma/nosepolyps/rhinosinusitis. Controls are all heathy subjects.

 

Question: Should basophil activation test (CD63, CD203c) be used to diagnose NSAID hypersensitivity in patients with NERD? (Bavbek, 2009)

Sensitivity

0.33

Specificity

0.79

 

Prevalences of NERD in specific populations

7%

3%

9%

 

 

Outcome

№ of studies (№ of patients)

Study design

Factors that may decrease quality of evidence

Effect per 1000 patients tested

Test accuracy QoE

Risk of bias

Indirectness

Inconsistency

Imprecision

Publication bias

pre-test probability of 7%

pre-test probability of 3%

pre-test probability of 9%

True positives
(patients with NSAID hypersensitivity)

1 study
42 patients

case-control type accuracy study

not serious

serious 1,2

not serious

serious 3

none

23

10

30

⨁⨁◯◯
LOW

False negatives
(patients incorrectly classified as not having NSAID hypersensitivity)

47

20

60

True negatives
(patients without NSAID hypersensitivity)

1 study
42 patients

case-control type accuracy study

not serious

serious 1,2

not serious

 serious 3

none

737

768

721

⨁⨁◯◯
LOW

False positives
(patients incorrectly classified as having NSAID hypersensitivity)

193

202

189

  1. Inclusion of healthy subjects into the control group.
  2. PICO question: the reference standard also included clinical history, not only a PT.
  3. Low number of subjects with NERD.
  4. Spectrum bias possible due to inclusion of healthy controls and case-control design. Healthy subjects would normally not be subjected to the index test.
  5. Inconsistensy could not be assessed.

 

Question: Should cellular allergen stimulation test (CAST) with Lys-ASA be used to diagnose NSAID hypersensitivity in adult patients with NERD? (Bavbek, 2009)

Sensitivity

0.25

Specificity

0.92

 

Prevalences of NERD in specific populations

7%

3%

9%

 

 

Outcome

№ of studies (№ of patients)

Study design

Factors that may decrease quality of evidence

Effect per 1000 patients tested

Test accuracy QoE

Risk of bias

Indirectness

Inconsistency

Imprecision

Publication bias

pre-test probability of 7%

pre-test probability of 3%

pre-test probability of 9%

True positives
(patients with NSAID hypersensitivity)

1 study
79 patients

case-control type accuracy study

serious 1

serious 2

not serious 3

serious 4

none

18

8

23

⨁◯◯◯
VERY LOW

False negatives
(patients incorrectly classified as not having NSAID hypersensitivity)

52

22

67

True negatives
(patients without NSAID hypersensitivity)

1 study
79 patients

case-control type accuracy study

serious 1

serious 2

not serious 3

serious 4

none

856

892

837

⨁◯◯◯
VERY LOW

False positives
(patients incorrectly classified as having NSAID hypersensitivity)

74

78

73

  1. Case-control study design, control group including healthy individuals.
  2. Inclusion of healthy subjects into the control group.
  3. Inconsistency could not be assessed.
  4. Low number of patients with and without NERD and 67% of the control group are healthy individuals.

 

Question: Should determination of eicosanoids in exhaled breath concentrates be used to diagnose NSAID hypersensitivity in adult patients with NERD? (Sanak, 2004; Sanak, 2011)

Sensitivity

0.99

Specificity

0.93

 

Prevalences of NERD in specific populations

7%

3%

9%

 

 

Outcome

№ of studies (№ of patients)

Study design

Factors that may decrease quality of evidence

Effect per 1000 patients tested

Test accuracy QoE

Risk of bias

Indirectness

Inconsistency

Imprecision

Publication bias

pre-test probability of 7%

pre-test probability of 3%

pre-test probability of 9%

True positives
(patients with NSAID hypersensitivity)

2 studies
197 patients

case-control type accuracy study

very serious

very serious 1

not serious 2

not serious

none

69

30

89

⨁◯◯◯
VERY LOW

False negatives
(patients incorrectly classified as not having NSAID hypersensitivity)

1

0

1

True negatives
(patients without NSAID hypersensitivity)

2 studies
197 patients

case-control type accuracy study

very serious

very serious 1

not serious 2

not serious

none

865

902

846

⨁◯◯◯
VERY LOW

False positives
(patients incorrectly classified as having NSAID hypersensitivity)

65

68

64

  1. Data of healthy subjects were used for the classification of disease status.
  2. Inconsistensy could not be assessed.
  3. The results for specificity were inconsistently reported between the 2 included studies.
  4. Patient selection and the execution of the index test might have introduced bias.

 

Question: Should determination of urinary eicosanoids (LTE4) be used to diagnose NSAID hypersensitivity in adult patients with NERD (cross-sectional)? (Celejewska-Wójcik, 2012)

Sensitivity

0.88

Specificity

0.94

 

Prevalences of NERD in specific populations

7%

3%

9%

 

 

Outcome

№ of studies (№ of patients)

Study design

Factors that may decrease quality of evidence

Effect per 1000 patients tested

Test accuracy QoE

Risk of bias

Indirectness

Inconsistency

Imprecision

Publication bias

pre-test probability of 7%

pre-test probability of 3%

pre-test probability of 9%

True positives
(patients with NSAID hypersensitivity)

1 studies
24 patients

cross-sectional (cohort type accuracy study)

not serious

not serious

not serious 1

serious 2

none

61

26

79

⨁⨁⨁◯
MODERATE

False negatives
(patients incorrectly classified as not having NSAID hypersensitivity)

5

2

6

True negatives
(patients without NSAID hypersensitivity)

1 studies
24 patients

cross-sectional (cohort type accuracy study)

not serious

not serious

not serious 1

serious 2

none

874

912

855

⨁⨁⨁◯
MODERATE

False positives
(patients incorrectly classified as having NSAID hypersensitivity)

46

48

46

  1. Inconsistensy could not be assessed.
  2. Small study population.

 

Question: Should determination of urinary eicosanoids (LTE4) be used to diagnose NSAID hypersensitivity in adult patients with NERD (Sanak, 2004)?

Sensitivity

0.93

Specificity

0.95

 

Prevalences of NERD in specific populations

7%

3%

9%

 

 

Outcome

№ of studies (№ of patients)

Study design

Factors that may decrease quality of evidence

Effect per 1000 patients tested

Test accuracy QoE

Risk of bias

Indirectness

Inconsistency

Imprecision

Publication bias

pre-test probability of 7%

pre-test probability of 3%

pre-test probability of 9%

True positives
(patients with NSAID hypersensitivity)

1 studies
44 patients

case-control type accuracy study

serious 1

serious 2

not serious 3

not serious

none

65

28

84

⨁⨁◯◯
LOW

False negatives
(patients incorrectly classified as not having NSAID hypersensitivity)

5

2

6

True negatives
(patients without NSAID hypersensitivity)

1 studies
44 patients

case-control type accuracy study

serious 1

serious 4

not serious 3

not serious

none

884

922

864

⨁⨁◯◯
LOW

False positives
(patients incorrectly classified as having NSAID hypersensitivity)

46

48

46

  1. Case-control design and inclusion of healthy volunteers.
  2. No explanation was provided.
  3. Inconsistensy could not be assessed.

 

Question: Should determination of urinary eicosanoids (LXA4) be used to diagnose NSAID hypersensitivity in adult patients with NERD (Yamaguchi, 2011)?

Sensitivity

0.85

Specificity

0.95

 

Prevalences of NERD in specific populations

7%

3%

9%

 

 

Outcome

№ of studies (№ of patients)

Study design

Factors that may decrease quality of evidence

Effect per 1000 patients tested

Test accuracy QoE

Risk of bias

Indirectness

Inconsistency

Imprecision

Publication bias

pre-test probability of 7%

pre-test probability of 3%

pre-test probability of 9%

True positives
(patients with NSAID hypersensitivity)

1 studies
41 patients

case-control type accuracy study

very serious 1

serious 2

not serious 3

not serious

none

60

26

77

⨁◯◯◯
VERY LOW

False negatives
(patients incorrectly classified as not having NSAID hypersensitivity)

10

4

13

True negatives
(patients without NSAID hypersensitivity)

1 studies
41 patients

case-control type accuracy study

very serious 1

serious 2

not serious 3

not serious

none

884

922

864

⨁◯◯◯
VERY LOW

False positives
(patients incorrectly classified as having NSAID hypersensitivity)

46

48

46

  1. Spectrum bias, case-control design, not all patients received the reference standard.
  2. PICO: intravenous DPT (not a golden standard in the Netherlands).
  3. Inconsistensy could not be assessed.

 

Question: Should (intra)nasal provocation test be used to diagnose NSAID hypersensitivity in adult patients with NERD? (Casadeval, 2000; Alonzo-Llamazares, 2002; White, 2006; Celikel, 2013; Miller, 2013)

Sensitivity

0.56 to 0.88

Specificity

0.64 to 0.94

 

Prevalences of NERD in specific populations

7%

3%

9%

 

 

Outcome

№ of studies (№ of patients)

Study design

Factors that may decrease quality of evidence

Effect per 1000 patients tested

Test accuracy QoE

Risk of bias

Indirectness

Inconsistency

Imprecision

Publication bias

pre-test probability of 7%

pre-test probability of 3%

pre-test probability of 9%

True positives
(patients with NSAID hypersensitivity)

5 studies
450 patients

case-control type accuracy study

very serious 1

serious 2

not serious 3

not serious

none

39 to 62

17 to 26

50 to 79

⨁◯◯◯
VERY LOW

False negatives
(patients incorrectly classified as not having NSAID hypersensitivity)

8 to 31

4 to 13

11 to 40

True negatives
(patients without NSAID hypersensitivity)

5 studies
450 patients

case-control type accuracy study

very serious 1

serious 2

not serious 3

not serious

none

595 to 874

621 to 912

582 to 855

⨁◯◯◯
VERY LOW

False positives
(patients incorrectly classified as having NSAID hypersensitivity)

56 to 335

58 to 349

55 to 328

  1. Inclusion of healthy subjects, choice and use of reference standard.
  2. Some of the studies included healthy volunteers into the control group. The study population differed in the number of patients with nose polyps.
  3. Inconsistensy could not be assessed.


Question: Should inhalation (bronchial) provocation test be used to diagnose NSAID hypersensitivity in adult patients with NERD? (Barranco, 2009; Nizankowska, 2000)

Sensitivity

0.70 to 0.77

Specificity

0.93 to 1.00

 

Prevalences of NERD in specific populations

7%

3%

9%

 

 

Outcome

№ of studies (№ of patients)

Study design

Factors that may decrease quality of evidence

Effect per 1000 patients tested

Test accuracy QoE

Risk of bias

Indirectness

Inconsistency

Imprecision

Publication bias

pre-test probability of 7%

pre-test probability of 3%

pre-test probability of 9%

True positives
(patients with NSAID hypersensitivity)

2 studies
78 patients

case-control type accuracy study

serious 1

serious 2

not serious 3

serious 4

none

49 to 54

21 to 23

63 to 69

⨁◯◯◯
VERY LOW

False negatives
(patients incorrectly classified as not having NSAID hypersensitivity)

16 to 21

7 to 9

21 to 27

True negatives
(patients without NSAID hypersensitivity)

2 studies
78 patients

case-control type accuracy study

serious 1

serious 2

not serious 3

serious 4

none

865 to 930

902 to 970

846 to 910

⨁◯◯◯
VERY LOW

False positives
(patients incorrectly classified as having NSAID hypersensitivity)

0 to 65

0 to 68

0 to 64

  1. One of the studies had a low risk of bias, while the other - high risk of bias due to the reference standard execution and patient flow.
  2. Healthy subjects were included into the control group of 1 of the studies.
  3. Inconsistensy could not be assessed.
  4. Low number of subjects and different reference tests were used in the 2 studies.