Study reference

Study characteristics

Patient characteristics

Index test

(test of interest)

Reference test

Follow-up

Outcome measures and effect size

Comments

Ferraro, 2013

PS., study characteristics and results are extracted from the SR (unless stated otherwise)

SR and meta-analysis

Literature search up to January 2012

A: Park, 2012

B: Bandiera, 2011

C: Escudero, 2011

D: Molina, 2011

E: Holcomb, 2011

F: Chang, 2011

G: Ruggeri, 2011

H: Jacob, 2011

I: Van Gorp, 2011

J: Montagnana, 2011

K: Yip, 2011

L: Modarres-Gilani, 2011

M: Abdel-Azeez, 2010

N: Anastasi, 2010

O: Nolen, 2010

Study design: case-control study (CC); cross-sectional study (CS); prospective clinical trial (PCT)

A: CS

B: CS

C: CC

D: CS

E: PCT

F: CS

G: CS

H: PCT

I: CC

J: PCT

K: CS

L: CC

M: CC

N: CS

O: CS

Setting and Country: not reported per included study.

Source of funding and conflicts of interest:

(commercial / non-commercial funding/ industrial co-authorship / potential conflicts of interest )

Not reported per individual study.

Ferraro (2013) reports no competing interest.

Inclusion criteria SR:

1. The primary or secondary aim of the study was at least the report of HE4 and CA-125 mean concentrations or sensitivity and specificity versus the ‘gold standard’ method for OC diagnosis—that is, laparoscopy with histological evaluation of biopsy material. 2. Diagnostic parameters were estimated according to a decisional threshold level and not to a fixed specificity or sensitivity.

3. The presentation of quantitative data allowed at least calculation of the OR.

4. The investigated population was represented by women with a gynaecological disease suspected as being OC, which is the intended spectrum of patients to be investigated by circulating biomarker detection.

5. HE4 concentrations were not included in diagnostic algorithms used to classify patients (incorporation bias)

Exclusion criteria SR:

Papers were excluded in the following instances:

1. Duplicative results from the same authors’ group were being reported. 2. Serum/plasma HE4 concentrations were measured to assess OC recurrence, to monitor disease progression or the effect of therapy.

3. They were case reports.

15 studies included (1 study, ref 33, was excluded by us because it included ovarian endometriosis only)

Important patient characteristics:

Number of patients; characteristics important to the research question; for example, age, sex, bmi, ...

N OC versus. BGD

A: 66 versus. 257

B: 113 versus. 165

C: 125 versus. 289

D: 111 versus. 285

E: 34 versus. 195

F: 52 versus. 150

G: 96 versus. 90

H: 29 versus. 71

I: 161 versus. 228

J: 55 versus. 40

K: 149 versus. 350

L: 37 versus. 50

M:41 versus. 24

N: 32 versus. 86

O: 227 versus. 158

Mean age: was not reported.

Sex: 100% female in all studies

Menopause status:

A: not reported

B:

OC: 26 pre-, 87 post-menopause; BGD: 69 pre-, 96 post-menopause

C: not reported.

D:

OC: 27 pre-, 84 post-menopause

BGD: 226 pre-, 59 post-menopause.

E:

OC: 34 pre-menopause

BGD: not reported

F: not reported.

G: not reported.

H: not reported.

I:

OC: 42 pre-, 119 post-menopause;

BGD: 142 pre-, 86 post-menopause

J: not reported.

K: not reported.

L: not reported.

M: not reported

N: not reported

O:

OC: 58 pre-, 169 post-menopause;

BGD: 18 pre-, 140 post-menopause.

Describe index and comparator tests* and

cut-off point(s):

Test 1: HE assay; cut-off (pmol/L)

Test 2: CA-125 assay; cut-off (kU/L)

A-1: test; cut-off: Abbott/Architect; 33.2

A-2: comp, cut-off:

Abbott/Architect; 38.3

B-1: test; cut-off: Abbott/Architect; pre-menopause: 70, post-menopause: 140

B-2: comp, cut-off:

Abbott/Architect; 35

C-1: test; cut-off: Abbott/Architect; 140

C-2: comp, cut-off:

Abbott/Architect; 35

D-1: test; cut-off: Abbott/Architect; 150

D-2: comp, cut-off:

Abbott/Architect; 35

E-1: test; cut-off: Abbott/Architect; 70

E-2: comp, cut-off:

Abbott/Architect; 35

F-1: test; cut-off: CanAg ELISA; 150

F-2: comp, cut-off:

CanAg ELISA; 35

G-1: test; cut-off: Abbott/Architect – Fujirebio ELISA; not reported

G-2: comp, cut-off:

Abbott/Architect – Fujirebio ELISA; not reported

H-1: test; cut-off: Fujirebio ELISA; 70

H-2: comp, cut-off:

Fujirebio ELISA; 35

I-1: test; cut-off: Fujirebio ELISA; 70 in pre-menopausal women; 150 in post-menopausal women

I-2: comp, cut-off:

Fujirebio ELISA; 35

J-1: test; cut-off: Fujirebio ELISA; 74.2

J-2: comp, cut-off:

Fujirebio ELISA; 35

K-1: test; cut-off: Luminex multiplexed ELISA; 54.8

K-2: comp, cut-off:

Luminex multiplexed ELISA; 52.2

L-1: test; cut-off: CanAg ELISA; 25

L-2: comp, cut-off:

CanAg ELISA; 37

M-1: test; cut-off: Fujirebio ELISA; 72

M-2: comp, cut-off:

Roche Diagnostics/ Cobas e411; 35

N-1: test; cut-off: Fujirebio ELISA; 150

N-2: comp, cut-off:

Radim RIA; 35

O-1: test; cut-off: Luminex multiplexed ELISA; 73.7

O-2: comp, cut-off:

Luminex multiplexed ELISA; 93.15

Describe reference test and cut-off point(s):

In all studies laparoscopy with histological evaluation of biopsy material was the golden standard.

Prevalence (%)

(based on refence test at specified cut-off point)

A: 20%

B: 40%

C: 30%

D: 28%

E: 15%

F: 26%

G: not reported

H: 29%

I: 41%

J: 53%

K: 30%

L: 43%

M: 63%

N: 27%

O: 59%

For how many participants were no complete outcome data available?

N (%)

Not reported per individual study.

Studies L and N were excluded from certain analyses due to outliers in data. Included studies reported per analysis.

Reasons for incomplete outcome data described?

Not reported per individual study.

Endpoint of follow-up:

Not reported per individual study.

Outcome measures and effect size (include 95%CI and p-value if available):

Outcome measure-1: sensitivity

See meta-analysis for sensitivity data per individual study (included studies were A,B,C,D,E,F,H,I,J,K,L,M,O).

Pooled characteristic (‘type of statistical analysis’ e.g. bivariate analysis) per index test and cut-off point:

Index test-1: HE4 (cut-off= reported per individual study)

79% (95% CI 76 to 81%); P=0.0001

Heterogeneity (reasons): 70.7% (due to the use of the same antibodies and to overlapping/harmonised thresholds).

Index test-2: CA-125 (cut-off= reported per individual study) 79% (95%CI 77 to 82%); P=0.0003

Heterogeneity (reasons): 66.6% (due to the use of the same antibodies and to overlapping/harmonised thresholds).

Outcome measure-2: specificity

See meta-analysis for specificity data per individual study.

(included studies were A,B,C,D,E,F,H,I,J,K, M,N,O).

Pooled characteristic (‘type of statistical analysis’ e.g. bivariate analysis) per index test and cut-off point:

Index test-1: HE4 (cut-off= reported per individual study)

93% (95% CI 92 to 94%); P<0.0001

Heterogeneity (reasons): 89.2% (due to the use of the same antibodies and to overlapping/harmonised thresholds).

Index test-2: CA-125 cut-off= reported per individual study)

78% (95%CI 76 to 80%); P<0.0001

Heterogeneity (reasons): 94.8% (due to the use of the same antibodies and to overlapping/harmonised thresholds).

Outcome measure-3: positive LR

See meta-analysis for specificity data per individual study.

(included studies were A,B,C,D,E,F,H,I,J,K,L,M,O).

Pooled characteristic (‘type of statistical analysis’ e.g. bivariate analysis) per index test and cut-off point:

Index test-1: HE4 cut-off= reported per individual study)

13.01 (95% CI 8.2 to 20.65); P<0.0001

Heterogeneity (reasons): 86.1% (reasons not reported).

Index test-2: CA-125 cut-off= reported per individual study)

4.15 (95%CI 3.06 to 5.62); P=0.0006

Heterogeneity (reasons): 91.8% (reasons not reported).

Outcome measure-4: negative LR

See meta-analysis for specificity data per individual study.

(HE4: included studies were A,B,C,D,E,F,H,I,J,K,M,O; CA-125: included studies were A,B,C,D,E,F,H,I,J,K,L,M,N,O).

Pooled characteristic (‘type of statistical analysis’ e.g. bivariate analysis) per index test and cut-off point:

Index test-1: HE4 (cut-off= reported per individual study)

0.23 (95% CI 0.19 to 0.28); P<0.0001

Heterogeneity (reasons): 65.1% (reasons not reported).

Index test-2: CA-125 (cut-off= reported per individual study)

0.27 (95%CI 0.23 to 0.31); P=0.0895

Heterogeneity (reasons): 36.7% (reasons not reported).

Positive (LR+) and negative (LR–) likelihood ratios, corresponding to sensitivity/(1−specificity) and (1−sensitivity)/ specificity

In particular, the strength of the indication for the presence of the disease provided by the positive result of the test is relevant when LR+≥10, modest when 5≤LR+<10, and poor when 2≤LR+<5, and the strength of the indication for the absence of the disease provided by the negative result of the test is relevant when LR–≤0.10, modest when 0.10<LR–≤0.20, and poor when 0.20<LR–≤0.50

PS. pooling of test accuracy data should be based on bivariate analysis of sensitivity and specificy; always mention the cut-off points used

Study quality (ROB): method used and results per individual study

QUADAS-II tool

Place of the index test in the clinical pathway: add-on to golden standard.

Choice of cut-off point: influences test characteristics (sens, spec); important in relation to the clinical question (e.g. if a disease is to be ruled out, sensitivity is the critical outcome measure and more important than specificity: high sensitivity comes at the expense of low specificity and high rates of false postives, and usually those testing positive are subjected to further diagnostic tests for final diagnosis)

Facultative:

Brief description of author’s conclusion:

HE4 measurement seems to be superior to CA-125 in terms of diagnostic performance for identification of OC in women with suspected gynaecological disease. Due to the high prevalence of OC in post-menopausal women and the need for data focused on early tumour stages, more studies tailored on these specific subsets are needed

Personal remarks on study quality, conclusions, and other issues (potentially) relevant to the research question

Sensitivity analyses (excluding small studies; excluding low quality studies; excluding case-control type of studies; relevant subgroup-analyses); mention only analyses which are of potential importance to the research question

Heterogeneity: clinical and statistical heterogeneity; clinical: enough similarities in patient characteristics, diagnostic tests (strategy) to allow pooling? For pooled data: assessment of statistical heterogeneity and, more importantly, assessment of the reasons for heterogeneity (if present)? Note: sensitivity and specificity depend on the situation in which the test is being used and the thresholds that have been set, and sensitivity and specificity are correlated; therefore, the use of heterogeneity statistics (p-values; I2) is problematic, and rather than testing whether heterogeneity is present, the reasons for heterogeneity should be examined.

Study reference

Study characteristics

Patient characteristics

Index test

(test of interest)

Reference test

Follow-up

Outcome measures and effect size

Comments

Huy, 2018

Type of study: cross-sectional study

Setting and country: Departments of Obstetrics and Gynecology, Hue University Hospital and Hue Central Hospital, Vietnam.

Funding and conflicts of interest: The authors whose names are listed above certify that they have NO affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers' bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patentlicensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript.

Inclusion criteria: women with clinically diagnosed ovarian tumours, including benign ovarian tumors and ovarian cancers. Patients with sufficient personal information, clinical symptoms, data on serum CA125 and serum HE4 levels, and postoperative pathologic findings were included. All patients were hospitalized for surgery for adnexal mass.

Exclusion criteria: Pregnant women, patients with a history of ovarian, primary peritoneal, or any other associated cancer were excluded from the study.

N=277 (247 benign, 30 EOC)

Prevalence: 10.8%

Age ≥ 50 years (n,%):

Benign: 37 (15%)

Malignant: 19 (63.3%)

Sex: 0% M / 100% F

Other important characteristics:

Postmenopausal women (n, %):

Benign: 30 (12.1%)

Malignant: 17 (56.7%)

Describe index test – HE4:

ARCHITECT i1000SR system respectively

Cut-off point(s):

Premenopausal women: >70 pmol/L

Postmenopausal women: >140 pmol/L

Describe index test – ROMA:

Algorithm was calculated as described by Moore et al. 2010, see text for formula.

Cut-off point(s):

Premenopausal women: ROMA values ≥7.4% or < 7.4% are regarded as high-risk and lowrisk respectively.

Postmenopausal women: ROMA values ≥25.3% or < 25.3% are regarded as high-risk and low-risk respectively.

Comparator test – CA-125:

Elecsys 2010 system immunoassay

Cut-off point(s): 35 U/mL

Describe reference test:

Postoperative pathological findings

Cut-off point(s): NA

Time between the index test en reference test: not reported. Blood was drawn preoperatively.

For how many participants were no complete outcome data available?

N (%) 0 (0%) patients with complete data included.

Reasons for incomplete outcome data described? NA

Outcome measures and effect size (include 95%CI and p-value if available):

Outcome measure 1: sensitivity

All patients (n=277)

Index test: HE4

50.0%

Index test: ROMA

80.0%

Comparator test: CA-125

83.3%

Pre-menopausal women only (n=230):

Index test: HE4

61.5%

Index test: ROMA

76.9%

Comparator test: CA-125

92.3%

Post-menopausal women only (n=47):

Index test: HE4

41.2%

Index test: ROMA

82.4%

Comparator test: CA-125

76.5%

Outcome measure 2: specificity

All patients (n=277)

Index test: HE4

98.38%

Index test: ROMA

84.6%

Comparator test: CA-125

78.5%

Pre-menopausal women only (n=230):

Index test: HE4

98.2%

Index test: ROMA

82.9%

Comparator test: CA-125

77.0%

Post-menopausal women only (n=47):

Index test: HE4

96.7%

Index test: ROMA

96.7%

Comparator test: CA-125

90.0%

Outcome measure 3: PPV

All patients (n=277)

Index test: HE4

78.95%

Index test: ROMA

38.71%

Comparator test: CA-125

31.65%

Pre-menopausal women only (n=230):

Index test: HE4

66.67%

Index test: ROMA

21.28%

Comparator test: CA-125

19.05%

Post-menopausal women only (n=47):

Index test: HE4

100%

Index test: ROMA

93.3%

Comparator test: CA-125

81.25%

Outcome measure 4: NPV

All patients (n=277)

Index test: HE4

94.19%

Index test: ROMA

97.21%

Comparator test: CA-125

97.47%

Pre-menopausal women only (n=230):

Index test: HE4

97.71%

Index test: ROMA

98.36%

Comparator test: CA-125

99.40

Post-menopausal women only (n=47):

Index test: HE4

75.0%

Index test: ROMA

90.62%

Comparator test: CA-125

87.10%

NB in abstract is stated that included women were hospitalized.

Liest, 2018

Type of study: prospective population-based study

Setting and country: the Department of Obstetrics and Gynecology, University Hospital, Linköping, Sweden, and from eight hospitals in the referral area, Sweden.

Funding and conflicts of interest:

FORSS (Medical Research Council of Southeast Sweden), County Council of Östergötland, and the local fund Östgötaregionens cancerfond.

Inclusion criteria: women aged ≥ 18 years with a pelvic mass of probable ovarian origin and scheduled for surgery. Both women with a high clinical suspicion of malignancy and those with probably benign tumors were included in the study

Exclusion criteria: none reported.

N= 784 (n=29 other malignancies; n=144 EOC including borderline, n=611 benign)

Prevalence (EOC): 18.4%

Mean age ± SD: not reported

Sex: 0% M / 100% F

Other important characteristics:

Postmenopausal (n, %):

EOC: 117 (81%)

Benign: 306 (50.1%)

Describe index test - HE4: electrochemiluminescence immunoassay on the automated cobas e602 module (Roche Diagnostics, Mannheim, Germany). The HE4 method has been standardized against the HE4 EIA method from Fujirebio Diagnostics (Gothenburg, Sweden). HE4 levels were measured in 279 urine samples by a quantitative enzyme‐linked immunosorbent assay (ELISA) kit from Fujirebio Diagnostics. Urine samples were diluted 1:40 with a tracer diluent provided with the kit prior to testing. Plates were read at 405 nm wavelength within 15 min using a Multiskan Spectrum microplate reader (Thermo Scientific, Vantaa, Finland).

Cut-off point(s):

2 methods: 1) determined by fixed specificity; 2) according to ROC.

Describe index test – ROMA: ROMA was calculated according to the algorithm proposed by Moore et al.

Cut-off point(s):

2 methods: 1) determined by fixed specificity; 2) according to ROC.

Comparator test – CA-125:

electrochemiluminescence immunoassay on the automated cobas e602 module (Roche Diagnostics, Mannheim, Germany). The CA125 method has been standardized against the Enzymun‐Test CA125 II method, which in turn has been standardized against the CA125 II RIA from Fujirebio Diagnostics (Gothenburg, Sweden)

Cut-off point(s): 2 methods: 1) determined by fixed specificity; 2) according to ROC.

Describe reference test: histopathology

Cut-off point(s): NA

Time between the index test en reference test: Blood samples for measuring CA125 and HE4 and urine samples for measuring HE4 were taken within 7 days prior to surgery

For how many participants were no complete outcome data available?

N (%) – NA, only patients with complete data selected.

Reasons for incomplete outcome data described? – NA only patients with complete data selected.

Outcome measures and effect size (include 95%CI and p-value if available):

Outcome measure 1: sensitivity

Premenopausal women only: (n=361)

Cut-off based on fixed specificity of 75%:

Index test: HE4 (cut-off: 59)

75%

Index test: ROMA (cut-off: 11)

80%

Comparator test: CA-125 (cut-off: 41)

95%

Cut-off based on ROC:

Index test: HE4 (cut-off: 63)

75%

Index test: ROMA (cut-off: 13)

75%

Comparator test: CA-125 (cut-off: 109)

65%

Postmenopausal women only: (n=423)

Cut-off based on fixed specificity of 75%:

Index test: HE4 (cut-off: 82)

82%

Index test: ROMA (cut-off: 25)

84%

Comparator test: CA-125 (cut-off: 28)

80%

Cut-off based on ROC:

Index test: HE4 (cut-off: 91)

77%

Index test: ROMA (cut-off: 35)

83%

Comparator test: CA-125 (cut-off: 42)

78%

Outcome measure 2: specificity

Premenopausal women only: (n=361)

Cut-off based on fixed specificity of 75%:

Index test: HE4 (cut-off: 59)

75%

Index test: ROMA (cut-off: 11)

75%

Comparator test: CA-125 (cut-off: 41)

75%

Cut-off based on ROC:

Index test: HE4 (cut-off: 63)

81%

Index test: ROMA (cut-off: 13)

83%

Comparator test: CA-125 (cut-off: 109)

93%

Postmenopausal women only: (n=423)

Cut-off based on fixed specificity of 75%:

Index test: HE4 (cut-off: 82)

75%

Index test: ROMA (cut-off: 25)

75%

Comparator test: CA-125 (cut-off: 28)

75%

Cut-off based on ROC:

Index test: HE4 (cut-off: 91)

82%

Index test: ROMA (cut-off: 35)

89%

Comparator test: CA-125 (cut-off: 42)

87%

Outcome measure 3: PPV

Premenopausal women only: (n=361)

Cut-off based on fixed specificity of 75%:

Index test: HE4 (cut-off: 59)

16%

Index test: ROMA (cut-off: 11)

17%

Comparator test: CA-125 (cut-off: 41)

20%

Cut-off based on ROC:

Index test: HE4 (cut-off: 63)

20%

Index test: ROMA (cut-off: 13)

22%

Comparator test: CA-125 (cut-off: 109)

36%

Postmenopausal women only: (n=423)

Cut-off based on fixed specificity of 75%:

Index test: HE4 (cut-off: 82)

46%

Index test: ROMA (cut-off: 25)

47%

Comparator test: CA-125 (cut-off: 28)

46%

Cut-off based on ROC:

Index test: HE4 (cut-off: 91)

53%

Index test: ROMA (cut-off: 35)

66%

Comparator test: CA-125 (cut-off: 42)

61%

Outcome measure 4: NPV

Premenopausal women only: (n=361)

Cut-off based on fixed specificity of 75%:

Index test: HE4 (cut-off: 59)

98%

Index test: ROMA (cut-off: 11)

98%

Comparator test: CA-125 (cut-off: 41)

100%

Cut-off based on ROC:

Index test: HE4 (cut-off: 63)

98%

Index test: ROMA (cut-off: 13)

98%

Comparator test: CA-125 (cut-off: 109)

98%

Postmenopausal women only: (n=423)

Cut-off based on fixed specificity of 75%:

Index test: HE4 (cut-off: 82)

94%

Index test: ROMA (cut-off: 25)

95%

Comparator test: CA-125 (cut-off: 28)

93%

Cut-off based on ROC:

Index test: HE4 (cut-off: 91)

93%

Index test: ROMA (cut-off: 35)

95%

Comparator test: CA-125 (cut-off: 42)

94%

Of the 901 women included in the study, 784 (87%) were evaluable (Figure 1). The major cause of exclusion was missing or incomplete data (13%).

Reasons:

65 Serum samples were taken but did not reach the laboratory

19 Serum samples were not obtained before surgery

25 Ovarian cysts had disappeared and no surgery was performed

6 Incomplete or failure in case report form

2 Withdrawal of informed consent

Oranratanphan, 2018

Type of study: prospective analytic study

Setting and country:

King Chulalongkorn Memorial Hospital, Thailand.

Funding and conflicts of interest: This research was supported by the Ratchadapiseksompotch Fund, Faculty of Medicine, Chulalongkorn University.

The authors declare that they have no conflicts of interest

Inclusion criteria: Women,aged above than 18 years, with nonfunctional ovarian cysts larger than3cminmaximaldiameter,scheduledforsurgery

Exclusion criteria: Patients with the previous history of ovarian cancer, primary peritoneal cancer or any known malignancies, previous bilateral oophorectomy, pregnancy, or functional cysts were excluded.

N= 281 (n=225 benign (80.1%); 56 malignancy (19.9%), of which n=26 early stage OC (9.2%), n=25 advanced-stage OC (8.9%), n=5 other malignancies (1.8%))

Prevalence (OC): 18.1%

Mean age ± SD: 44 ± 13.16 years

Sex: 0% M / 100% F

Other important characteristics:

Post-menopause (n, %): 68 (24.2%)

Describe index test – HE4: electrochemiluminescence immunoassay (ECLIA) with the same

operator and the same machine (MODULAR ANALYTICS E170, cobas e601 and cobas e602 analyzers)

Cut-off point(s): >70 pMol/L and >150 pMol/L

Describe index test – ROMA: The ROMA score was calculated automatically by the computer program with a standard formula

Cut-off point(s): not specified for pre- and postmenopause, but shown separately.

Comparator test – CA-125:

electrochemiluminescence immunoassay (ECLIA) with the same

operator and the same machine (MODULAR ANALYTICS E170, cobas e601 and cobas e602 analyzers)

Cut-off point(s): >35 IU/mL

Describe reference test: pathology report

Cut-off point(s): NA

Time between the index test en reference test: before surgery, timing not specified.

For how many participants were no complete outcome data available?

N (%): not reported, presumably only patients with complete data were selected.

Reasons for incomplete outcome data described? NA

Outcome measures and effect size (include 95%CI and p-value if available):

Outcome measure 1: sensitivity

Index test: HE4 (cut-off: >150 pMol/L)

53.4%

Index test: HE4 (cut-off: >70 pMol/L)

74.1%

Index test: ROMA (pre-menopause)

66.67%

Index test: ROMA (post-menopause)

88.89%

Comparator test: CA-125 (>35 IU/mL)

87.9%

Outcome measure 2: specificity

Index test: HE4 (cut-off: >150 pMol/L)

97.8%

Index test: HE4 (cut-off: >70 pMol/L)

86.5%

Index test: ROMA (pre-menopause)

88.527%

Index test: ROMA (post-menopause)

70%

Comparator test: CA-125 (>35 IU/mL)

46.2%

Outcome measure 3: PPV

Index test: HE4 (cut-off: >150 pMol/L):

86.1%

Index test: HE4 (cut-off: >70 pMol/L)

59.2%

Index test: ROMA (pre-menopause)

48.78%

Index test: ROMA (post-menopause)

66.67%

Comparator test: CA-125 (>35 IU/mL)

29.8%

Outcome measure 4: NPV

Index test: HE4 (cut-off: >150 pMol/L)

89%

Index test: HE4 (cut-off: >70 pMol/L)

93.3%

Index test: ROMA (pre-menopause)

94.197%

Index test: ROMA (post-menopause)

90.92%

Comparator test: CA-125 (>35 IU/mL)

93.6%

Pitta, 2013

Type of study: cross-sectional study

Setting and country: Unicamp school of medicine, Brazil.

Funding and conflicts of interest: no competing interests declared

Inclusion criteria: women with adnexal masses of suspected ovarian origin.

Exclusion criteria: women with previous operation for adnexal mass and ongoing pregnancy.

N=176 adnexal mass (116 benign, 60 malignant)

Prevalence (among N total for benign and malignant tumours): 34.1%

Mean age ± SD:

Benign: 47.9 ± 16.6

Malignant: 50.8 ± 20.5

Sex: 0% M / 100% F

Other important characteristics:

Postmenopausal women (n, %):

Benign: 49 (42.%)

Malignant: 38 (61.7%)

Histology (n, %):

Benign:

Epithelial 40 (34.5%), germ line 28 (24.1%); sex cord and stromal 20 (17.3%); non-neoplastic 28 (24.1%).

Malignant: epithelial 47 (78.3%); germ line 6 (10.0%), sex cord and stromal 7 (12.7%)

Describe index test: ROMA: ARCHITECT CA125 II assay, which is a Chemiluminescent Microparticle Immunoassay (CMIA)

HE4: HE4 enzyme immunometric assay Kits (EIA) (Fujirebio Diagnostics), based on the direct sandwich technique, solid-phase immunoassay according to the manufacturers’ instructions.

Cut-off point(s):

ROMA:

Premenopausal women: 5.01%

Postmenopausal women: 18.2%

HE4:

Premenopausal women: 41.6 pmol/L

Postmenopausal women: 96.6 pmol/L

Comparator test:

CA-125: solid phase chemiluminescence using OM-MA test. Immunochemiluminometric assay (ICMA), Immulite 2000 OM-MA, Siemens Medical Solutions Diagnostics.

Cut-off point(s):

Premenopausal women: 69.8 U/L

Postmenopausal women: 21.7 U/L

Describe reference test: histopathological diagnosis of surgical specimen

Cut-off point(s): NA.

Time between the index test en reference test: not reported. Blood was drawn before surgery.

For how many participants were no complete outcome data available?

N (%): not reported.

Reasons for incomplete outcome data described? Not reported.

Outcome measures and effect size (include 95%CI and p-value if available):

Outcome measure 1: sensitivity

Index test: HE4

58.3% (95% CI 49.4 to 67.3%)

Index test: ROMA

56.7% (95% CI 47.6 to 65.7%)

Comparator test: CA-125

65.0% (95%CI 56.3 to 73.7%)

Outcome measure 2: specificity

Index test: HE4

84.5% (95% CI 74.7 to 94.2%)

Index test: ROMA

87.9% (95% CI 78.7 to 97.1%)

Comparator test: CA-125

75.9% (95%CI 65.6 to 86.1%)

Outcome measure 3: PPV

Index test: HE4

66.0%

Index test: ROMA

70.8%

Comparator test: CA-125

58.2%

Outcome measure 4: NPV

Index test: HE4

79.9%

Index test: ROMA

79.9%

Comparator test: CA-125

80.7%

Borderline tumours were classified as malignant: 10 out of 47 epithelial malignant tumours were rendered as borderline).

Stiekema, 2014

Type of study:

Retrospective observational study

Setting and country: Netherlands Cancer Institute, Netherlands.

Funding and conflicts of interest: there were no conflicts of interest.

Inclusion criteria: all women with histologically confirmed epithelial ovarian cancer or benign ovarian disease who were referred to the Netherlands Cancer institute.

Exclusion criteria: Borderline tumours and/or patients with insufficient clinical data to establish disease stage were excluded.

N= 181 (34 (18.8%) benign disease; 147 (81.2%) malignant disease).

Prevalence EOC (excluding other malignancies than EOC): 73.5%

Mean age ± SD:

Benign: 57 (SD not reported)

OC: 58 (SD not reported)

Sex: 0% M / 100% F

Other important characteristics:

Postmenopausal (%):

Benign: 77%

OC: 75%

Histology (n,%):

Benign disease:

Serous cysteadenoma 9 (27%); Mucinous cysteadenoma 13 (38%); Cysteadenoma NOS 3 (9%); Othert than adenoma 3 (9%);

Missing 6 (18%)

OC:

Papillary serous 72(46%); Mucinous 4 (3%); Clear cell 7 (5%); Endometrioïd 25 (16.0%); Adenocarcinoma NOS 30 (29%) ; Other than EOC 9 (6%).

Describe index test – HE4: ARCHITECT i1000SRr

Cut-off point(s): 70 pmol/L

Describe index test – ROMA: as suggested by Moore et al.

Cut-off point(s): pre-menopausal 0.129; post-menopausal 0.278

Comparator test - CA-125:

Cut-off point(s): 35 kU/L

Describe reference test: histopathology

Cut-off point(s): NA

Time between the index test and reference test: not reported.

For how many participants were no complete outcome data available? N (%) Not reported, presumably patients with complete data were selected.

Reasons for incomplete outcome data described? NA

Outcome measures and effect size (include 95%CI and p-value if available):

Outcome measure 1: sensitivity

Pre-menopausal women only (n=38)

Index test: HE4 (cut-off: 70 pmol/L)

0.84 (95% CI 0.67 to 0.95)

Index test: ROMA (cut-off: 0.129)

0.84 (95% CI 0.67 to 0.95)

Comparator test: CA-125 (cut-off: 35 kU/L)

0.91 (95% CI 0.75 to 0.98)

Post-menopausal women only (n=143)

Index test: HE4 (cut-off: 70 pmol/L)

0.91 (95% CI 0.84 to 0.96)

Index test: ROMA (cut-off: 0.278)

0.91 (95% CI 0.84 to 0.96)

Comparator test: CA-125 (cut-off: 35 kU/L)

0.91 (95% CI 0.84 to 0.96)

Outcome measure 2: specificity

Pre-menopausal women only (n=38)

Index test: HE4 (cut-off: 70 pmol/L)

1.00 (95% CI 0.63 to 1.00)

Index test: ROMA (cut-off: 0.129)

1.00 (95% CI 0.63 to 1.00)

Comparator test: CA-125 (cut-off: 35 kU/L)

0.62 (95% CI 0.24 to 0.91)

Post-menopausal women only (n=143)

Index test: HE4 (cut-off: 70 pmol/L)

0.96 (95% CI 0.80 to 1.00)

Index test: ROMA (cut-off: 0.278)

0.77 (95% CI 0.56 to 0.91)

Comparator test: CA-125 (cut-off: 35 kU/L)

0.65 (95% CI 0.44 to 0.83)

180 healthy controls were recruited in the study. Data not reported here.

Study

First author, year

Appropriate and clearly focused question?¹

Yes/no/unclear

Comprehensive and systematic literature search?²

Yes/no/unclear

Description of included and excluded studies?³

Yes/no/unclear

Description of relevant characteristics of included studies?⁴

Yes/no/unclear

Assessment of scientific quality of included studies?⁵

Yes/no/unclear

Enough similarities between studies to make combining them reasonable?⁶

Yes/no/unclear

Potential risk of publication bias taken into account?⁷

Yes/no/unclear

Potential conflicts of interest reported?⁸

Yes/no/unclear

Ferraro, 2013

Yes, to critically revise the available literature on the comparison between the diagnostic accuracy of HE4 and carbohydrate antigen 125 (CA-125) to confirm the additional clinical value of HE4.

Yes

Yes, see supplementary file 1 on website of journal with reasons for exclusion

Yes, although mean age was not reported per study.

Yes, QUADAS-II tool was used. Overall quality of evidence was low and authors therefore conclude that confidence in effect estimates are therefore limited.

Unclear, only 8 of 16 included studies concern target population (women with pelvic mass undergoing marker determination for diagnostic purposes).

Yes

No, not reported for the individual studies.

Study reference

Patient selection

Index test

Reference standard

Flow and timing

Comments with respect to applicability

Huy, 2018

Was a consecutive or random sample of patients enrolled?

Unclear

Was a case-control design avoided?

Yes

Did the study avoid inappropriate exclusions?

Yes

Were the index test results interpreted without knowledge of the results of the reference standard?

Unclear

If a threshold was used, was it pre-specified?

Yes

Is the reference standard likely to correctly classify the target condition?

Yes

Were the reference standard results interpreted without knowledge of the results of the index test?

Unclear

Was there an appropriate interval between index test(s) and reference standard?

Unclear

Did all patients receive a reference standard?

Yes

Did patients receive the same reference standard?

Yes

Were all patients included in the analysis?

Yes, because only patients with complete data were selected.

Are there concerns that the included patients do not match the review question?

No

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

No

Are there concerns that the target condition as defined by the reference standard does not match the review question?

Yes, type of cysts not specified.

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: HIGH, only patients with complete data selected.

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW, by using a prespecified cut-off bias is unlikely.

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW, presumably pathologist was not aware of tumor marker values.

CONCLUSION

Could the patient flow have introduced bias?

RISK: LOW

Liest, 2018

Was a consecutive or random sample of patients enrolled?

No, non-randomized

Was a case-control design avoided?

Yes

Did the study avoid inappropriate exclusions?

Yes

Were the index test results interpreted without knowledge of the results of the reference standard?

Unclear

If a threshold was used, was it pre-specified?

Yes, two thresholds were studied (fixed and based on ROC)

Is the reference standard likely to correctly classify the target condition?

Yes

Were the reference standard results interpreted without knowledge of the results of the index test?

Unclear

Was there an appropriate interval between index test(s) and reference standard?

Unclear

Did all patients receive a reference standard?

Yes, all included patients

Did patients receive the same reference standard?

Yes

Were all patients included in the analysis?

Yes, because only patients with complete data were selected.

Are there concerns that the included patients do not match the review question?

Yes, because type of cysts were not described.

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

No

Are there concerns that the target condition as defined by the reference standard does not match the review question?

Yes, because type of cysts were not described.

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: HIGH, only people with complete data included, reasons for inclusion were reported and could have introduced selection bias.

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW, by using a prespecified cut-off bias is unlikely.

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW, presumably pathologist was not aware of tumor marker values.

CONCLUSION

Could the patient flow have introduced bias?

RISK: LOW

Oranratanphan, 2018

Was a consecutive or random sample of patients enrolled?

Unclear

Was a case-control design avoided?

Yes

Did the study avoid inappropriate exclusions?

Yes

Were the index test results interpreted without knowledge of the results of the reference standard?

Unclear

If a threshold was used, was it pre-specified?

Yes, except for ROMA which was not specified.

Is the reference standard likely to correctly classify the target condition?

Yes

Were the reference standard results interpreted without knowledge of the results of the index test?

Unclear

Was there an appropriate interval between index test(s) and reference standard?

Unclear

Did all patients receive a reference standard?

Yes

Did patients receive the same reference standard?

Yes

Were all patients included in the analysis?

Yes, because only patients with complete data were selected.

Are there concerns that the included patients do not match the review question?

No

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

No

Are there concerns that the target condition as defined by the reference standard does not match the review question?

Yes, because type of cysts were not described.

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: HIGH, only patients with complete data selected.

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: UNCLEAR, unknown cut-off used for ROMA

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW, presumably pathologist was not aware of tumor marker values.

CONCLUSION

Could the patient flow have introduced bias?

RISK: LOW

Pitta, 2013

Was a consecutive or random sample of patients enrolled?

Unclear

Was a case-control design avoided?

Yes

Did the study avoid inappropriate exclusions?

Yes

Were the index test results interpreted without knowledge of the results of the reference standard?

Unclear

If a threshold was used, was it pre-specified?

Yes

Is the reference standard likely to correctly classify the target condition?

Yes

Were the reference standard results interpreted without knowledge of the results of the index test?

Unclear

Was there an appropriate interval between index test(s) and reference standard?

Unclear

Did all patients receive a reference standard?

Yes

Did patients receive the same reference standard?

Yes

Were all patients included in the analysis?

Yes, because only patients with complete data were selected.

Are there concerns that the included patients do not match the review question?

No

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

No

Are there concerns that the target condition as defined by the reference standard does not match the review question?

Yes, possibly patients with e.g. simple cysts included which should be excluded according to our PICO.

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: HIGH, only people with complete data included, reasons for inclusion were reported and could have introduced selection bias.

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW, by using a prespecified cut-off bias is unlikely.

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW, presumably pathologist was not aware of tumor marker values.

CONCLUSION

Could the patient flow have introduced bias?

RISK: LOW,

Stiekema, 2014

Was a consecutive or random sample of patients enrolled?

Unclear

Was a case-control design avoided?

Yes

Did the study avoid inappropriate exclusions?

Yes

Were the index test results interpreted without knowledge of the results of the reference standard?

Unclear

If a threshold was used, was it pre-specified?

Yes

Is the reference standard likely to correctly classify the target condition?

Yes

Were the reference standard results interpreted without knowledge of the results of the index test?

Unclear

Was there an appropriate interval between index test(s) and reference standard?

Unclear

Did all patients receive a reference standard?

Yes

Did patients receive the same reference standard?

Yes

Were all patients included in the analysis?

Yes, because presumably only patients with complete data were selected.

Are there concerns that the included patients do not match the review question?

No

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

No

Are there concerns that the target condition as defined by the reference standard does not match the review question?

No

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: HIGH, only people with complete data included.

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW, by using a prespecified cut-off bias is unlikely.

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW, presumably pathologist was not aware of tumor marker

values.

CONCLUSION

Could the patient flow have introduced bias?

RISK: LOW

(first author, year of publication)

Was a consecutive or random sample of patients enrolled?

Yes/No/Unclear

Was a case-control design avoided?

Yes/No/Unclear

Did the study avoid inappropriate exclusions?

Yes/No/Unclear

Were the index test results interpreted without knowledge of the results of the reference standard?

Yes/No/Unclear

If a threshold was used, was it pre-specified?

Yes/No/Unclear

Is the reference standard likely to correctly classify the target condition?

Yes/No/Unclear

Were the reference standard results interpreted without knowledge of the results of the index test?

Yes/No/Unclear

Was there an appropriate interval between index test(s) and reference standard?

Yes/No/Unclear

Did all patients receive a reference standard?

Yes/No/Unclear

Did patients receive the same reference standard?

Yes/No/Unclear

Were all patients included in the analysis?

Yes/No/Unclear

Are there concerns that the included patients do not match the review question?

Yes/No/Unclear

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

Yes/No/Unclear

Are there concerns that the target condition as defined by the reference standard does not match the review question?

Yes/No/Unclear

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: LOW/HIGH/UNCLEAR

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW /HIGH/UNCLEAR

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW /HIGH/UNCLEAR

CONCLUSION

Could the patient flow have introduced bias?

RISK: LOW /HIGH/UNCLEAR

(first author, year of publication)

Was a consecutive or random sample of patients enrolled?

Yes/No/Unclear

Was a case-control design avoided?

Yes/No/Unclear

Did the study avoid inappropriate exclusions?

Yes/No/Unclear

Were the index test results interpreted without knowledge of the results of the reference standard?

Yes/No/Unclear

If a threshold was used, was it pre-specified?

Yes/No/Unclear

Is the reference standard likely to correctly classify the target condition?

Yes/No/Unclear

Were the reference standard results interpreted without knowledge of the results of the index test?

Yes/No/Unclear

Was there an appropriate interval between index test(s) and reference standard?

Yes/No/Unclear

Did all patients receive a reference standard?

Yes/No/Unclear

Did patients receive the same reference standard?

Yes/No/Unclear

Were all patients included in the analysis?

Yes/No/Unclear

Are there concerns that the included patients do not match the review question?

Yes/No/Unclear

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

Yes/No/Unclear

Are there concerns that the target condition as defined by the reference standard does not match the review question?

Yes/No/Unclear

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: LOW/HIGH/UNCLEAR

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW /HIGH/UNCLEAR

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW /HIGH/UNCLEAR

CONCLUSION

Could the patient flow have introduced bias?

RISK: LOW /HIGH/UNCLEAR