Study reference

Study characteristics

Patient characteristics

Intervention (I)

Comparison / control (C)

Follow-up

Outcome measures and effect size

Comments

Castellana, 2020

SR and meta-analysis of retrospective cohort study

Literature search up to March 2019

A: Negro, 2017

B: Yoon, 2017

C: Ha, 2018

D: Ha, 2018

E: Middleton,

2018

F: Persichetti,

2018

G: Xu, 2018

H: Grani, 2019

I: Mohammadi,

2019

J: Ruan, 2019

K: Trimboli,

2019

L: Wu, 2019

Study design: retrospective cohort studies

Setting and Country:

Hospital, worldwide

Source of funding and conflicts of interest:

None.

Inclusion criteria SR: studies reporting the diagnostic performance of at least 1 of

the US RSSs in thyroid nodules were included if meeting both

of the following criteria: (i) the diagnosis of benign nodules

was based either on histology or core-needle biopsy (CNB)

or cytology, and (ii) the diagnosis of malignant nodules was

not based on cytology only

Exclusion criteria SR:

Studies focusing on pediatric patients or specific subgroups of

thyroid nodules (ie, indeterminate), as well as studies using cytology

as the only reference standard for both malignant and

benign nodules

12 studies included

Important patient characteristics at baseline:

included studies had sample sizes ranges from 424 to 4696 thyroid nodules. 4378 malignant and 14 372 benign nodules

were included in the present review.

Prevalence ranges from 4% to 54%

N, mean age

n.a.

Sex:

n.a.

Groups comparable at baseline?

No, stratified analyses are performed

Describe intervention:

A: AACE

B: ATA

C: ACR-TIRADs, ATRA, K-TIRADs

D: AACE, ACR-TIRADs, ATRA, K-TIRADs

E: ACR-TIRADs, ATRA, K-TIRADs

F: AACE, ATA

G: all TIRADs

H: all test

I:ATA

J: ACR-TIRADs, ATA

K: EU-TIRADs

L : ACR-TIRADs

Describe control:

histology, cytology (FNA) all studies

End-point of follow-up:

N.a.

For how many participants were no complete outcome data available?

n.a. retrospective.

If data was missing, authors were contacted via email.

Outcome measure-1

Defined as ACR-TIRADs

Effect measure: sensitivity [95% CI]:

Grani,2019= 0.83 (0.67-0.94)

ha, 2018= 0.82 (0.76-0.86)

ha, 2018=0.75 (0.70-0.79)

middelton, 2018= 0.68 (0.63-0.73)

ruan, 2019=0.53 (0.48-0.58)

wu, 2019= 0.91 (0.88-0.94)

xu, 2018= 0.49 (0.46-0.52)

Effect measure: specificity [95% CI]:

Grani,2019= 0.56 (0.52-0.61)

ha, 2018= 0.69 (0.65-0.72)

ha, 2018= 0.67 (0.65-0.70)

middelton, 2018= 0.63 (0.51-0.55)

ruan, 2019= 0.77 (0.73-0.80)

wu, 2019= 0.49 (0.43-0.55)

xu, 2018= 0.71 (0.68-0.73)

Facultative:

Brief description of author’s conclusion

Personal remarks on study quality, conclusions, and other issues (potentially) relevant to the research question

Level of evidence: GRADE (per comparison and outcome measure) including reasons for down/upgrading

Sensitivity analyses (excluding small studies; excluding studies with short follow-up; excluding low quality studies; relevant subgroup-analyses); mention only analyses which are of potential importance to the research question

Heterogeneity: clinical and statistical heterogeneity; explained versus unexplained (subgroupanalysis)

Castellana, 2020

Yes, The systematic review was registered on PROSPERO (registration

number CRD42019131771) and performed in accordance

with the Preferred Reporting Items for a Systematic

Review and Meta-analysis of Diagnostic Test Accuracy Studies

(PRISMA-DTA)

Yes, PubMed,

CENTRAL, Scopus, and Web of Science were searched.

Yes, this is described in figure 1.

Yes, this is described in table 1.

Yes, stratified analyses were performed per diagnostic test.

Yes, the Quality

Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool

Yes, stratified analyses were performed

No, Publication bias was not evaluated, because of uncertainty about the determinants

for diagnostic accuracy studies and the inadequacy of tests for

detecting funnel plot asymmetry

Yes, there are no conflicts of interest.

Gacayan, 2021

Type of study^[1]:

cross-sectional criterion-referenced study

Setting and country:

Hospital, Philippines

Funding and conflicts of interest:

None.

Inclusion criteria:

Filipino patients with thyroid nodules

aging 18 to 80 years old who underwent ultrasound

guided fine needle aspiration biopsy of thyroid nodules at

the Thyroid Clinic of The Medical City from July 2019 to

December 2019.

Exclusion criteria:

the cytology report

of the FNAB is inadequate or non-diagnostic.

N=121 (197 nodules)

Prevalence: 81%

Median age ± range: 53 (21-77)

Sex: 15% M / 85% F

Other important characteristics:

-

Describe index test:

ACR- TIRADs

Cut-off point(s):

Appendix A

Comparator test^[2]:

fine needle aspiration

biopsy (FNAB)

Cut-off point(s):

Bethesda Classification

Describe reference test^[3]:

Histology

Cut-off point(s):
Papillary thyroid carcinoma

Multinodular colloid goiter

Follicular thyroid carcinoma

Time between the index test en reference test:

n.a.

For how many participants were no complete outcome data available?

n.a. retrospective

Reasons for incomplete outcome data described?

n.a. retrospective

Outcome measures and effect size (include 95%CI and p-value if available)⁴:

TIRADs ACR

Sensitivity: 100% (80.5 to 100)

Specificity: 52.2% (44.7 to 59.7)

PPV: 16.5% (14.5 to 18.7)

NPV: 100%

No information about histology/pathology of all patients > overestimation

Relative small sample size.

Paker, 2021

Type of study:

Retrospective cohort study

Setting and country:

Hospital,

Israel

Funding and conflicts of interest:

Not mentioned

Inclusion criteria:

medical records of patients who had undergone partial (lobectomy)

or subtotal or total thyroidectomy between January

2012 and March 2019. Notably, 2012 was the first year that

physicians in the department began practicing FNA guided

by ultrasound of thyroid nodules.

Exclusion criteria:

thyroidectomies performed due to non-thyroid

disease and non-differentiated thyroid lesions (lymphoma, anaplastic,

amyloidosis).

N=305 (338 nodules)

*final cohort

N=216 (238 nodules)

Prevalence: 48.3%

Mean age ± SD: 50.0 ± 12.4

Sex: 80 % F

Other important characteristics:

-

Describe index test:

TIRADS

Cut-off point(s):

See article.

Comparator test:

FNAB

Cut-off point(s):

 See article

Describe reference test:

Histology after surgery.

Cut-off point(s):

n.a.

Time between the index test en reference test:

N.a.

For how many participants were no complete outcome data available?

N.a. retrospective

Reasons for incomplete outcome data described?
89 patients were excluded from the analysis for the following reasons: no preoperative ultrasound was performed at our center (n=42); no nodules were observed (Grave's disease, enlarged thyroid lobe, completion thyroidectomy)

(n=19); there were more than four nodules (Hashimoto

thyroiditis, multinodular goiter) (n=18); thyroidectomy was performed due to non-thyroid disease (such as parathyroidectomy) (n=7); non-differentiated thyroid lesions presented (lymphoma,

anaplastic, amyloidosis) (n=3).

Outcome measures and effect size (include 95%CI and p-value if available):

TIRADs ACR

Sensitivity: 0.843

Specificity 0.536

PPV: 0.629

NPV: 0.785

No information about histology/pathology of all patients > overestimation

Relative small sample size.

Seminati, 2021

Type of study:

Prospective cohort study

Setting and country:

Hospital, Italy

Funding and conflicts of interest:

This work was funded by the grant Ricerca Finalizzata 2019-GR-2019-12368592.

Inclusion criteria:

patients who underwent 493 USguided

FNA from January to June 2019 at the interventional radiology clinic, ASST Monza,

Italy, during an Italian Association for Research on Cancer (AIRC)-granted project for the

diagnosis of thyroid carcinoma [8]. All nodules were subjected to FNA, regardless of their

ACR/EU-TIRADS scores, after an endocrinological clinical indication

Exclusion criteria:

n.a.

N= 448 (480 nodules)

Prevalence: 87.1%

Mean age ± SD:

Not mentioned

Sex: % M / % F

Not mentioned

Other important characteristics:

-

Describe index test:

TIRADS

Cut-off point(s):

See article.

Comparator test:

FNAB

Cut-off point(s):

 See article

Describe reference test:

Histology after surgery.

Cut-off point(s):

n.a.

Time between the index test en reference test:

n.a.

For how many participants were no complete outcome data available?

N (%)

n.a.

Reasons for incomplete outcome data described?

Na.

Outcome measures and effect size (include 95%CI and p-value if available):

TIRADs ACR

Sensitivity: 67.6% (49.5 to 82.6)

Specificity: 57.2% (52.0 to 62.3)

PPV: 12.8% (8.3 to 18.6)

NPV: 95.0% (91.3 to 97.5)

Histology was available in 49 resected nodules.

The ROM was estimated according to histological evaluation or

US follow-up examination.

McClean, 2021

Type of study:

Retrospectively

Setting and country:

Hospital, UK

Funding and conflicts of interest:
none.

Inclusion criteria:

patients were selected for surgery based on clinical

assessment and FNA result. From 2014, patients were selected for

surgery according to BTA guidelines.

Exclusion criteria:
Patients who underwent FNA

and surgery without US assessment were not included in the study.

US assessments performed outside of our institution and where the

reports were not available were excluded from the study.

N= 296 (308 nodules)

Prevalence: 43.8%

Mean age: 49

Sex: 77.3% M

Other important characteristics:

-

Describe index test:

ACR TIRADs

Cut-off point(s):

See article

Comparator test:

FNA

Cut-off point(s):

 See article

Describe reference test:

Histology

Cut-off point(s):

N.a.

Time between the index test en reference test:

n.a.

For how many participants were no complete outcome data available?

N (%)

N.a. retrospective

Reasons for incomplete outcome data described?
N.a. retrospective

Outcome measures and effect size (include 95%CI and p-value if available):

ACR TIRADs

Sensi: 73.3% (65 to 80.6)

Speci: 64.2% (56.5 to 71.3)

Ppv: 61.5% (53.5 to 69)

Npv: 75.5% (67.7 to 82.2)

Data collection was

performed by one author, who was blinded to the final histology of

the nodule.

Sparano, 2021

Type of study:

Retrospective cohort

Setting and country:

tertiary Endocrinology

outpatient clinics, Italy

Funding and conflicts of interest:
none.

Inclusion criteria:

all consecutive adult subjects (i.e., age > 18 years) for whom

fine-needle aspiration (FNA) was indicated, and who provided

a written informed consent.

Exclusion criteria:

Non-diagnostic cytology

and nodules with clinical or incomplete US assessments

were not included in this study. In addition, nodules with a

size lower than 10 mm were also excluded from the analysis,

considering that most of the available scores do not routinely

recommend FNA for sub-centimeter thyroid nodules

N= 6401 (6474 nodules)

Prevalence: 5%

Mean age ± SD: not mentioned

Sex: 78% F

Other important characteristics:

Describe index test:

TIRADs ACR

Cut-off point(s):

See supplementary files.

Comparator test:

FNAB

Cut-off point(s):

 See supplementary files

Describe reference test:

Histology

Cut-off point(s):

N.a.

Time between the index test en reference test:

n.a.

For how many participants were no complete outcome data available?

N (%)

n.a.

Reasons for incomplete outcome data described?

Na.

Outcome measures and effect size (include 95%CI and p-value if available):

TIRADs ACR

Sensitivity: 72.7%

Specificity: 31.7%

PPV: 8.9% (7.9 to 9.9)

NPV: 93.0% (91.7 to 94.3)

* spared FNA 1324 (20.6%)

Retrospective design.

Watkins, 2021

Type of study:

Retrospective cohort study.

Setting and country:

Hospital, UK

Funding and conflicts of interest:

Not mentioned.

Inclusion criteria:

patients in a large health board who had undergone pre-operative

thyroid ultrasound with eutopic thyroid histology

results available between January 1, 2017 and December 31,

2017,

Exclusion criteria:

due to ultrasound demonstrating diffuse thyroid disease such as

thyroiditis or diffuse multinodular goitre rather than a discrete

nodule (14) or if it was not considered possible to reliably

correlate imaging and histopathology, due to, e.g. suboptimal

image quality (11).

N= 212 (215 nodules)

Prevalence:35.3%

Mean age ± SD: 58.5 (29)

Sex: 76% F

Other important characteristics:
-

Describe index test:

ACR tirads

Cut-off point(s):

See article

Comparator test:

FNA

Cut-off point(s):

 See article.

Describe reference test:

Histology

Cut-off point(s):
n.a.

Time between the index test en reference test:

n.a.

For how many participants were no complete outcome data available?

N (%)

n.a. retrospective

Reasons for incomplete outcome data described?

N.a. retrospective

Outcome measures and effect size (include 95%CI and p-value if available):

ACR TIRADs

Seni: 95.24% (86.71 to 99.01)

Spec 40.57 (31.13 to 50.54)
PPV 48.78 (44.63 to 52.94)
NPV 93.48 (82.27 to 97.79)

Histology was available in resected nodules.

Relative small sample size.

Pandya, 2019

Type of study:

Retrospective cohort study

Setting and country:

Hospital, USA

Funding and conflicts of interest: none.

Inclusion criteria:

All subjects undergoing first-time FNA of a thyroid nodule in

the radiology department at our institution between October

2009 and February 2016 were identified via the electronic

medical record system and Department of Radiology records.

From this group, 28 patients had undergone repeat procedural

visits for FNA of a thyroid nodule, and thus only the most recent

procedure was included

Exclusion criteria:

For patients who underwent FNA of

several nodules in one procedural visit, only the first nodule

sampled was included in the data. There were no other exclusion criteria.

N= 1947 nodules

Prevalence: 6.2%

Mean age ± SD: 56 (15)

Sex: 76 % F

Other important characteristics:

-

Describe index test:

ACR tirads

Cut-off point(s):

See article

Comparator test:

FNA

Cut-off point(s):

 See article.

Describe reference test:

Histology

Cut-off point(s):
n.a.

Time between the index test en reference test:

n.a.

For how many participants were no complete outcome data available?

N (%)

n.a. retrospective cohort study

Reasons for incomplete outcome data described?

Outcome measures and effect size (include 95%CI and p-value if available):

ACR TIRADs

Seni: 75%
spec 41.6%
PPV 11.1 %
NPV: 94.5%

Ahmadi, 2018

Type of study:

Retrospective cohort study

Setting and country:

Hospital, USA

Funding and conflicts of interest:

Not mentioned.

Inclusion criteria:

thyroid ultrasound imaging from 213 adult patients (323 nodules) with thyroid nodules >5 mm who underwent thyroid surgery at a tertiary care hospital

Exclusion criteria:

more than four nodules, no discrete thyroid nodule, pre-op ultrasound imaging not available, and poor quality ultrasound imaging.

N= 213 (323 noduli(

Prevalence: 27.2%

Median; 55 years.

Sex: 76 % F

Other important characteristics:

-

Describe index test:

ACR tirads

Cut-off point(s):

See article

Comparator test:

FNA

Cut-off point(s):

 See article.

Describe reference test:

Histology

Cut-off point(s):
n.a.

Time between the index test en reference test:

n.a.

For how many participants were no complete outcome data available?

N (%)

n.a. retrospective cohort study

Reasons for incomplete outcome data described?

Outcome measures and effect size (include 95%CI and p-value if available):

ACR TIRADs

Seni: 78.4% (69.8-87.0)
spec 73.2% (67.5-78.9)
PPV 53.3% (43.7-60.8)
NPV: 90.1% (85.8 -94.3)

All ultrasound features were evaluated retrospectively at a single institution, where the incidence of malignancy was relatively high.

final histopathology to confirm malignancy.

De Melo, 2022

Type of study:

Retrospective cohort study

Setting and country:

Cancer Center, Brazil

Funding and conflicts of interest:

none

Inclusion criteria:

Patients who have undergo FNAB in centre.

Exclusion criteria:

US images were not available or did not allow a proper retrospective classification.

N= 803 (1112 noduli)

Prevalence: ?  in sub population 65.8%

Mean age ± SD: 52 years

Sex: 80 % F

Other important characteristics:

-

Describe index test:

ACR tirads

Cut-off point(s):

See article

Comparator test:

FNA

Cut-off point(s):

 See article.

Describe reference test:

Histology

Cut-off point(s):
n.a.

Time between the index test en reference test:

Yes

For how many participants were no complete outcome data available?

N (%)

n.a. retrospective study

Reasons for incomplete outcome data described?

Outcome measures and effect size (include 95%CI and p-value if available):

Sensi: 40% (33 to 47)

Speci:77% (67 to 87)

PPV: 82% (74 to 90)

NPV:32% (44 to 56)

Diagnostic values based on patients who underwent surgery; n = 117 (237 noduli)

Araruna Bezerra de Melo R, Menis F, Calsavara VF, Stefanini FS, Novaes T, Saieg M. The impact of the use of the ACR-TIRADS as a screening tool for thyroid nodules in a cancer center. Diagn Cytopathol. 2022 Jan;50(1):18-23. doi: 10.1002/dc.24904. Epub 2021 Nov 19. PMID: 34797612.

Soylemez, 2021

Type of study:

Prospective study

Setting and country: Hospital, Turkey

Funding and conflicts of interest:

None.

Inclusion criteria:

US was performed on all patients

referred to our Interventional Radiology Department for

thyroid biopsy. The Vision Preirus system (Hitachi

Medical Corp.) and Aplio 500 device (Toshiba Medical

Systems, Co., Ltd.) were used for all biopsy and US

procedures.

Exclusion criteria:

Patients aged <18 years were not

included in the study. Nodules with non-diagnostic

(Bethesda 1) cytology results were also excluded.

N= 1010 nodules

Prevalence:7.8% (Based on 939 noduli)

Mean age ± SD: 52 (13)

Sex: 82 % F

Other important characteristics:

-

Describe index test:

ACR tirads

Cut-off point(s):

See article

Comparator test:

FNA

Cut-off point(s):

 See article.

Describe reference test:

Histology (of 68 patients who underwent surgery)

Cut-off point(s):
n.a.

Time between the index test en reference test:

yes

For how many participants were no complete outcome data available?

N 71 noduli (7%)

Reasons for incomplete outcome data described?

Not descripted

Outcome measures and effect size (include 95%CI and p-value if available):

All

sensi

0,945         0,865-0,984

speci

0,457         0,423-0,491

Malignancy

Sensi

0,947         0,822-0,994

speci

0,8             0,614-0,923

1-3 cm noduli

sensi

0,913         0,792-0,975

speci

0,457         0,423-0,491

Histology was available in 68 resected nodules.

Barbosa, 2019

Type of study:

Retrospective cohort study

Setting and country:

Hospital, Brazil

Funding and conflicts of interest:

None.

Inclusion criteria:

(a) indeterminate cytology; (b) a

thyroid US image and (c) surgical resection with a histopathological

result matching with the nodule’s location and

size analyzed on US-FNAC.

Exclusion criteria:

lack or

absence of information on the US, FNAC or histology.

N=139 (140 noduli)

Prevalence: 47%

Mean age ± SD: 49 ± 13

Sex: 85 % F

Other important characteristics:

-

Describe index test:

ACR tirads

Cut-off point(s):

See article

Comparator test:

FNA

Cut-off point(s):

 See article.

Describe reference test:

Histology (of 68 patients who underwent surgery)

Cut-off point(s):
n.a.

Time between the index test en reference test:

n.a. retrospective

For how many participants were no complete outcome data available?

N (%)

n.a. retrospective

Reasons for incomplete outcome data described?

Outcome measures and effect size (include 95%CI and p-value if available):

TIRADs

Sensi: 69.7%(57 to 80)

Speci: 81.1% (70 to89)

PPV: 76.7%(67 to 84)

NPV: 75% (67 to 81)

Gacayan, 2021

Was a consecutive or random sample of patients enrolled?

Yes, all patients who were undergoing ultrasound guided

FNA were asked to participate in the study with

a signed informed consent.

Was a case-control design avoided?

Yes, cross-sectional criterion

Did the study avoid inappropriate exclusions?

Yes, if the cytology report

of the FNAB is inadequate or non-diagnostic.

Were the index test results interpreted without knowledge of the results of the reference standard?

Yes, first TIRADs

If a threshold was used, was it pre-specified?

Yes, appendix A

Is the reference standard likely to correctly classify the target condition?

Yes, histology/ FNAB

Were the reference standard results interpreted without knowledge of the results of the index test?

Unclear

Was there an appropriate interval between index test(s) and reference standard?

Yes

Did all patients receive a reference standard?

Unclear

Did patients receive the same reference standard?

No, not all patients received histology

Were all patients included in the analysis?

Yes

Are there concerns that the included patients do not match the review question?

No, all patients with suspected DTC.

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

No, index test was performed at first.

Are there concerns that the target condition as defined by the reference standard does not match the review question?

Unclear, not all patients received this test.

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: LOW

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: UNCLEAR

CONCLUSION

Could the patient flow have introduced bias?

RISK: HIGH

Paker, 2021

Was a consecutive or random sample of patients enrolled?

Yes, from database.

Was a case-control design avoided?

Yes, retrospective cohort study.

Did the study avoid inappropriate exclusions?

Yes, thyroidectomies performed due to non-thyroid

disease and non-differentiated thyroid lesions (lymphoma, anaplastic,

amyloidosis).

Were the index test results interpreted without knowledge of the results of the reference standard?

Yes, blind to FNA

If a threshold was used, was it pre-specified?

Unclear, not in detail.

Is the reference standard likely to correctly classify the target condition?

Yes, histology/ FNAB

Were the reference standard results interpreted without knowledge of the results of the index test?

Yes

Was there an appropriate interval between index test(s) and reference standard?

Yes

Did all patients receive a reference standard?

unclear

Did patients receive the same reference standard?

No, not all patients received histology

Were all patients included in the analysis?

Yes

Are there concerns that the included patients do not match the review question?

No, all patients with suspected DTC.

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

No

Are there concerns that the target condition as defined by the reference standard does not match the review question?

Unclear, not all patients received this test.

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: LOW

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW

CONCLUSION

Could the patient flow have introduced bias?

RISK: HIGH

Seminati, 2021

Was a consecutive or random sample of patients enrolled?

Yes, patients who underwent 493 USguided FNA from January to June 2019 at the interventional radiology clinic, ASST Monza, Italy, during an Italian Association for Research on Cancer (AIRC)-granted project for the diagnosis of thyroid carcinoma

Was a case-control design avoided?

Yes, data prospectively collected.

Did the study avoid inappropriate exclusions?

Unclear, exclusion criteria not mentioned.

Were the index test results interpreted without knowledge of the results of the reference standard?

Unclear

If a threshold was used, was it pre-specified?

Unclear, not in deatail

Is the reference standard likely to correctly classify the target condition?

Yes, histology/ FNAB

Were the reference standard results interpreted without knowledge of the results of the index test?

Unclear

Was there an appropriate interval between index test(s) and reference standard?

Yes,

Did all patients receive a reference standard?

Unclear

Did patients receive the same reference standard?

No, not all patients received histology

Were all patients included in the analysis?

Yes,

Are there concerns that the included patients do not match the review question?

No

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

No

Are there concerns that the target condition as defined by the reference standard does not match the review question?

Yes

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: LOW

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: UNCLEAR

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: UNCLEAR

CONCLUSION

Could the patient flow have introduced bias?

RISK: HIGH

Sparano, 2021

Was a consecutive or random sample of patients enrolled?

Yes, all consecutive adult subjects (i.e., age > 18 years) for whom

fine-needle aspiration (FNA) was indicated, and who provided

a written informed consent.

Was a case-control design avoided?

Yes

Did the study avoid inappropriate exclusions?

Yes, Non-diagnostic cytology

and nodules with clinical or incomplete US assessments

were not included in this study. In addition, nodules with a

size lower than 10 mm were also excluded from the analysis,

considering that most of the available scores do not routinely

recommend FNA for sub-centimeter thyroid nodules.

Were the index test results interpreted without knowledge of the results of the reference standard?

Yes, blinded

If a threshold was used, was it pre-specified?

Yes, see article

Is the reference standard likely to correctly classify the target condition?

Yes, histology/ FNAB

Were the reference standard results interpreted without knowledge of the results of the index test?

Yes

Was there an appropriate interval between index test(s) and reference standard?

Yes

Did all patients receive a reference standard?

Unclear

Did patients receive the same reference standard?

No, not all patients received histology

Were all patients included in the analysis?

Yes

Are there concerns that the included patients do not match the review question?

No

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

No

Are there concerns that the target condition as defined by the reference standard does not match the review question?

Yes

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: LOW

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW

CONCLUSION

Could the patient flow have introduced bias?

RISK: HIGH

McClean, 2021

Was a consecutive or random sample of patients enrolled?

Yes, patients were selected for surgery based on clinical

assessment and FNA result.

Was a case-control design avoided?

Yes, retrospective cohort.

Did the study avoid inappropriate exclusions?

Yes, Patients who underwent FNA

and surgery without US assessment were not included in the study.

US assessments performed outside of our institution and where the

reports were not available were excluded from the study.

Were the index test results interpreted without knowledge of the results of the reference standard?

Yes, blinded

If a threshold was used, was it pre-specified?

Yes, see article

Is the reference standard likely to correctly classify the target condition?

Yes, histology

Were the reference standard results interpreted without knowledge of the results of the index test?

Yes, blinded

Was there an appropriate interval between index test(s) and reference standard?

Yes

Did all patients receive a reference standard?

Yes

Did patients receive the same reference standard?

Yes

Were all patients included in the analysis?

Yes

Are there concerns that the included patients do not match the review question?

No

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

No

Are there concerns that the target condition as defined by the reference standard does not match the review question?

No

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: LOW

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW

CONCLUSION

Could the patient flow have introduced bias?

RISK: LOW

Watkins, 2021

Was a consecutive or random sample of patients enrolled?

Yes, patients in a large health board who had undergone pre-operative

thyroid ultrasound with eutopic thyroid histology

results

Was a case-control design avoided?

Yes

Did the study avoid inappropriate exclusions?

Yes, patients were excluded

due to ultrasound demonstrating diffuse thyroid disease such as

thyroiditis or diffuse multinodular goitre rather than a discrete

nodule (14) or if it was not considered possible to reliably

correlate imaging and histopathology, due to, e.g. suboptimal

image quality (11)

Were the index test results interpreted without knowledge of the results of the reference standard?

Yes

If a threshold was used, was it pre-specified?

Yes, see article

Is the reference standard likely to correctly classify the target condition?

Yes, histology

Were the reference standard results interpreted without knowledge of the results of the index test?

No,

Was there an appropriate interval between index test(s) and reference standard?

Yes

Did all patients receive a reference standard?

Yes

Did patients receive the same reference standard?

Yes

Were all patients included in the analysis?

Yes

Are there concerns that the included patients do not match the review question?

No

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

no

Are there concerns that the target condition as defined by the reference standard does not match the review question?

No

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: LOW

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW

CONCLUSION

Could the patient flow have introduced bias?

RISK: LOW

Ahmadi, 2019

Was a consecutive or random sample of patients enrolled?

Yes, retrospectively from database.

Was a case-control design avoided?

Yes

Did the study avoid inappropriate exclusions?

Yes

Were the index test results interpreted without knowledge of the results of the reference standard?

No, reference outcome was known

If a threshold was used, was it pre-specified?

Yes

Is the reference standard likely to correctly classify the target condition?

Yes

Were the reference standard results interpreted without knowledge of the results of the index test?

Yes

Was there an appropriate interval between index test(s) and reference standard?

No, first reference, later index

Did all patients receive a reference standard?

Yes

Did patients receive the same reference standard?

Yes

Were all patients included in the analysis?

Yes

Are there concerns that the included patients do not match the review question?

No

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

Yes

Are there concerns that the target condition as defined by the reference standard does not match the review question?

No

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: LOW

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: High

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW

CONCLUSION

Could the patient flow have introduced bias?

RISK: High

Pandya, 219

Was a consecutive or random sample of patients enrolled?

Yes, retrospectively from database.

Was a case-control design avoided?

Yes

Did the study avoid inappropriate exclusions?

Yes

Were the index test results interpreted without knowledge of the results of the reference standard?

Yes, reviewers were blinded

If a threshold was used, was it pre-specified?

Yes

Is the reference standard likely to correctly classify the target condition?

Yes, reviewers were blinded

Were the reference standard results interpreted without knowledge of the results of the index test?

Yes

Was there an appropriate interval between index test(s) and reference standard?

Yes

Did all patients receive a reference standard?

Unclear

Did patients receive the same reference standard?

Unclear.

Were all patients included in the analysis?

Yes

Are there concerns that the included patients do not match the review question?

No

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

No

Are there concerns that the target condition as defined by the reference standard does not match the review question?

Unclear

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: LOW

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW

CONCLUSION

Could the patient flow have introduced bias?

RISK: UNCLEAR

De Melo, 2022

Was a consecutive or random sample of patients enrolled?

Unclear, only diagnostic values of a selection

Was a case-control design avoided?

Yes

Did the study avoid inappropriate exclusions?

No

Were the index test results interpreted without knowledge of the results of the reference standard?

Yes

If a threshold was used, was it pre-specified?

Yes

Is the reference standard likely to correctly classify the target condition?

Yes

Were the reference standard results interpreted without knowledge of the results of the index test?

Yes

Was there an appropriate interval between index test(s) and reference standard?

Yes

Did all patients receive a reference standard?

No, only selection

Did patients receive the same reference standard?

Yes

Were all patients included in the analysis?

Yes

Are there concerns that the included patients do not match the review question?

No

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

No

Are there concerns that the target condition as defined by the reference standard does not match the review question?

No

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: UNCLEAR

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW

CONCLUSION

Could the patient flow have introduced bias?

RISK: HIGH

Soylemez, 2021

Was a consecutive or random sample of patients enrolled?

Yes

Was a case-control design avoided?

Yes

Did the study avoid inappropriate exclusions?

Yes

Were the index test results interpreted without knowledge of the results of the reference standard?

Yes

If a threshold was used, was it pre-specified?

Yes

Is the reference standard likely to correctly classify the target condition?

Yes

Were the reference standard results interpreted without knowledge of the results of the index test?

Yes

Was there an appropriate interval between index test(s) and reference standard?

Yes

Did all patients receive a reference standard?

No

Did patients receive the same reference standard?

Unclear

Were all patients included in the analysis?

Unclear

Are there concerns that the included patients do not match the review question?

No

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

No

Are there concerns that the target condition as defined by the reference standard does not match the review question?

No

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: LOW

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW

CONCLUSION

Could the patient flow have introduced bias?

RISK: UNCLEAR

Was a consecutive or random sample of patients enrolled?

Yes/No/Unclear

Was a case-control design avoided?

Yes/No/Unclear

Did the study avoid inappropriate exclusions?

Yes/No/Unclear

Were the index test results interpreted without knowledge of the results of the reference standard?

Yes/No/Unclear

If a threshold was used, was it pre-specified?

Yes/No/Unclear

Is the reference standard likely to correctly classify the target condition?

Yes/No/Unclear

Were the reference standard results interpreted without knowledge of the results of the index test?

Yes/No/Unclear

Was there an appropriate interval between index test(s) and reference standard?

Yes/No/Unclear

Did all patients receive a reference standard?

Yes/No/Unclear

Did patients receive the same reference standard?

Yes/No/Unclear

Were all patients included in the analysis?

Yes/No/Unclear

Are there concerns that the included patients do not match the review question?

Yes/No/Unclear

Are there concerns that the index test, its conduct, or interpretation differ from the review question?

Yes/No/Unclear

Are there concerns that the target condition as defined by the reference standard does not match the review question?

Yes/No/Unclear

CONCLUSION:

Could the selection of patients have introduced bias?

RISK: LOW/HIGH/UNCLEAR

CONCLUSION:

Could the conduct or interpretation of the index test have introduced bias?

RISK: LOW /HIGH/UNCLEAR

CONCLUSION:

Could the reference standard, its conduct, or its interpretation have introduced bias?

RISK: LOW /HIGH/UNCLEAR

CONCLUSION

Could the patient flow have introduced bias?

RISK: LOW /HIGH/UNCLEAR

Referentie

Reden voor exclusie

Jabar ASS, Koteshwara P, Andrade J. Diagnostic reliability of the Thyroid Imaging Reporting and Data System (TI-RADS) in routine practice. Pol J Radiol. 2019 Jun 10;84:e274-e280. doi: 10.5114/pjr.2019.86823. PMID: 31482001; PMCID: PMC6717936.

No histology

Basha MAA, Alnaggar AA, Refaat R, El-Maghraby AM, Refaat MM, Abd Elhamed ME, Abdalla AAEM, Aly SA, Hanafy AS, Mohamed AEM, Afifi AHM, Harb O. The validity and reproducibility of the thyroid imaging reporting and data system (TI-RADS) in categorization of thyroid nodules: Multicentre prospective study. Eur J Radiol. 2019 Aug;117:184-192. doi: 10.1016/j.ejrad.2019.06.015. Epub 2019 Jun 18. PMID: 31307646.

No histology

Jabar ASS, Koteshwara P, Andrade J. Diagnostic reliability of the Thyroid Imaging Reporting and Data System (TI-RADS) in routine practice. Pol J Radiol. 2019 Jun 10;84:e274-e280. doi: 10.5114/pjr.2019.86823. PMID: 31482001; PMCID: PMC6717936.

Not according to PICO

Thai JN, Peti S, Hayim M, Sharma R, Koshy J, Demissie S, Sarkany D, Raden M, Scheiner J, Lin C. Substantial interreader agreement for biopsy with reduction in biopsy rate: A multireader diagnostic performance study of ACR TI-RADS. Clin Imaging. 2022 Apr;84:93-97. doi: 10.1016/j.clinimag.2022.02.002. Epub 2022 Feb 10. PMID: 35158125.

Not according to PICO

Grani G, Brenta G, Trimboli P, Falcone R, Ramundo V, Maranghi M, Lucia P, Filetti S, Durante C. Sonographic Risk Stratification Systems for Thyroid Nodules as Rule-Out Tests in Older Adults. Cancers (Basel). 2020 Aug 30;12(9):2458. doi: 10.3390/cancers12092458. PMID: 32872622; PMCID: PMC7564359.

not according to PICO

Grimmichová T, Pačesová P, Srbová L, Vrbíková J, Havrdová T, Hill M. The gold standard of thyroid nodule examination? Prospective validation of the ACR TI-RADS in a secondary referral center. Physiol Res. 2020 Sep 30;69(Suppl 2):S329-S337. doi: 10.33549/physiolres.934515. PMID: 33094631; PMCID: PMC8603730.

not according to PICO

Huang BL, Ebner SA, Makkar JS, Bentley-Hibbert S, McConnell RJ, Lee JA, Hecht EM, Kuo JH. A Multidisciplinary Head-to-Head Comparison of American College of Radiology Thyroid Imaging and Reporting Data System and American Thyroid Association Ultrasound Risk Stratification Systems. Oncologist. 2020 May;25(5):398-403. doi: 10.1634/theoncologist.2019-0362. Epub 2019 Nov 19. PMID: 31740569; PMCID: PMC7216459.

no references standard

Koc AM, Adıbelli ZH, Erkul Z, Sahin Y, Dilek I. Comparison of diagnostic accuracy of ACR-TIRADS, American Thyroid Association (ATA), and EU-TIRADS guidelines in detecting thyroid malignancy. Eur J Radiol. 2020 Dec;133:109390. doi: 10.1016/j.ejrad.2020.109390. Epub 2020 Nov 2. PMID: 33181485.

no references standard

Chen X, Kutaiba N, Pearce S, Digby S, Van Gelderen D. Application of Thyroid Imaging Reporting and Data System (TIRADS) guidelines to thyroid nodules with cytopathological correlation and impact on healthcare costs. Intern Med J. 2022 Aug;52(8):1366-1373. doi: 10.1111/imj.15343. Epub 2022 Jun 1. PMID: 33942959.

histology not reference

Freire da Silva P, Corrêa de Araújo Arcoverde L, de Siqueira Barbosa Arcoverde L, Tenório Wanderley Fernandes Lima G, Paes de Medeiros Lima T, José do Amaral F, Bandeira F. Agreement Between American and European Thyroid Imaging, Reporting, and Data System (TIRADS) in the Diagnosis of 473 Thyroid Nodules From a Single Center in Brazil. Endocr Pract. 2021 Nov;27(11):1108-1113. doi: 10.1016/j.eprac.2021.06.008. Epub 2021 Jun 18. PMID: 34153510.

O not inline PICO

Hoang JK, Middleton WD, Langer JE, Schmidt K, Gillis LB, Nair SS, Watts JA, Snyder RW 3rd, Khot R, Rawal U, Tessler FN. Comparison of Thyroid Risk Categorization Systems and Fine-Needle Aspiration Recommendations in a Multi-Institutional Thyroid Ultrasound Registry. J Am Coll Radiol. 2021 Dec;18(12):1605-1613. doi: 10.1016/j.jacr.2021.07.019. Epub 2021 Aug 20. PMID: 34419476.

Histology not reference

Huh S, Yoon JH, Lee HS, Moon HJ, Park VY, Kwak JY. Comparison of diagnostic performance of the ACR and Kwak TIRADS applying the ACR TIRADS' size thresholds for FNA. Eur Radiol. 2021 Jul;31(7):5243-5250. doi: 10.1007/s00330-020-07591-1. Epub 2021 Jan 15. PMID: 33449191.

Not according to PICO

Merhav G, Zolotov S, Mahagneh A, Malchin L, Mekel M, Beck-Razi N. Validation of TIRADS ACR Risk Assessment of Thyroid Nodules in Comparison to the ATA Guidelines. J Clin Imaging Sci. 2021 Jul 10;11:37. doi: 10.25259/JCIS_99_2021. PMID: 34345527; PMCID: PMC8326070.

histology not reference

Liu J, Guo Y, Xiao J, Chen L, Liang Z. Comparison of the Efficacy and Safety of the American Thyroid Association Guidelines and American College of Radiology TI-RADS. Endocr Pract. 2021 Jul;27(7):661-667. doi: 10.1016/j.eprac.2020.11.013. Epub 2020 Dec 15. PMID: 34250908.

histology not reference