Utility of pneumonia severity assessment tools for mortality prediction in healthcare-associated pneumonia: a systematic review and meta-analysis

Database search and risk of bias assessment

A total of 2881 articles (PubMed 2276, Cochrane Library 134, and Ichushi web 471) were identified in the initial search, and 41 articles were potentially eligible after the first screening of the titles and abstracts. Next, the full text was reviewed, and 20 articles were excluded. Eventually, 21 observational studies were selected for this study (Fig. 1). The summary of the risk of bias using the QUADAS-2 in the included studies was shown in Fig. 2. For patient selection, three or two studies were evaluated as having a high risk of bias or high concern for applicability, respectively, because of the possibility of inappropriate exclusion or mismatched definition. In the patient flow and timing assessment, three studies were assessed to have a high risk of bias because of inappropriate omission or uncertainly of evaluation timing of reference standard.

Flow chart of the study selection.

Summary of risk of bias using QUADAS-2.

Included studies

The characteristics of the included studies are shown in Table 2. Among these 21 studies, five^{6,13,18,30,36} and 16^{4,7,14,19,31,32,33,34,35,37,38,39,40,41,42,43} were prospective and retrospective studies, respectively. The category of pneumonia was eight^{7,19,31,33,36,38,39,42}, seven^{13,18,32,34,35,41,43}, and six^{4,6,14,30,37,40} for HCAP, NHCAP, and NHAP, respectively. Twelve^{4,7,13,14,19,30,31,36,39,41,42,43} studies for PSI, 12^{4,7,13,18,19,32,33,34,35,38,41,43} for A-DROP, seven^{7,13,18,19,34,41,43} for I-ROAD, and 14^{4,6,7,13,14,19,30,31,36,37,39,40,41,42} for CURB-65 were included in this study.

PSI

Twelve^{4,7,13,14,19,30,31,36,39,41,42,43} studies were included in the meta-analysis for the PSI score. Using a cut-off value of ≥ IV (moderate; n = 12), the pooled sensitivity, specificity, PLR, NLR, and DOR for mortality were calculated as 0.97 (0.94–0.98), 0.15 (0.10–0.21), 1.14 (1.08–1.20), 0.22 (0.12–0.38), and 5.09 (2.95–8.78), respectively (Table 3). Using a cut-off value of V (severe; n = 11), the pooled sensitivity, specificity, PLR, NLR, and DOR for mortality were 0.69 (0.60–0.77), 0.66 (0.60–0.72), 2.03 (1.82–2.27), 0.47 (0.38–0.58), and 4.32 (3.35–5.59), respectively. The forest plots and estimated sensitivities and specificities from each study are shown in Fig. 3a,b.

The paired forest plots of sensitivity and specificity for predicting mortality with PSI, A-DROP, I-ROAD, and CURB-65. Forest plots of sensitivity and specificity for mortality prediction with (a) PSI ≥ IV, (b) PSI V, (c) A-DROP ≥ I, (d) A-DROP ≥ III, (e) I-ROAD ≥ moderate, (f) I-ROAD ≥ severe, (g) CURB-65 ≥ II, and (h) CURB-65 ≥ III.

A-DROP

Twelve^{4,7,13,18,19,32,33,34,35,38,41,43} studies were included in the meta-analysis for the A-DROP score. Using a cut-off value of ≥ III (severe; n = 11), the pooled sensitivity, specificity, PLR, NLR, and DOR for mortality were 0.70 (0.62–0.76), 0.54 (0.45–0.62), 1.50 (1.33–1.70), 0.56 (0.48–0.66), and 2.66 (2.09–3.40), respectively (Table 3). The forest plots using these cut-offs and estimated sensitivities and specificities from each study are shown in Fig. 3c,d. In one study⁷, forest plots weren’t described in Figure because the data necessary to create it in both a cut-off value of ≥ I and ≥ III was insufficient.

I-ROAD

Seven^{7,13,18,19,34,41,43} studies were included in the meta-analysis for the I-ROAD score. Using a cut-off value of ≥ moderate (n = 5), the pooled sensitivity, specificity, PLR, NLR, and DOR for mortality were 0.92 (0.69–0.98), 0.44 (0.30–0.59), 1.66 (1.39–1.98), 0.18 (0.05–0.61), and 9.32 (2.86–30.3) respectively (Table 3). Using a cut-off value of severe (n = 7), the pooled sensitivity, specificity, PLR, NLR, and DOR for mortality were 0.67 (0.54–0.77), 0.63 (0.50–0.74), 1.78 (1.44–2.21), 0.53 (0.42–0.68), and 3.34 (2.35–4.75), respectively. The forest plots using these cut-offs and estimated sensitivities and specificities from each study are shown in Fig. 3e,f.

CURB-65

Fourteen^{4,6,7,13,14,19,30,31,36,37,39,40,41,42} studies were included in the meta-analysis for the CURB-65. Using a cut-off value of ≥ II (moderate; n = 13), the pooled sensitivity, specificity, PLR, NLR, and DOR for mortality were 0.91 (0.84–0.95), 0.28 (0.20–0.37), 1.26 (1.17–1.36), 0.33 (0.23–0.46), and 3.86 (2.74–5.44), respectively (Table 3). Using a cut-off value of ≥ III (severe; n = 14), the pooled sensitivity, specificity, PLR, NLR, and DOR for mortality were 0.63 (0.52–0.73), 0.63 (0.53–0.71), 1.70 (1.52–1.90), 0.58 (0.49–0.70), and 2.91 (2.34–3.62), respectively. The forest plots using these cut-offs and estimated sensitivities and specificities from each study are shown in Fig. 3g,h.

Comparisons of overall AUC among PSI, A-DROP, I-ROAD, and CURB-65

The overall AUC values were pooled from the AUC (95% CI) values reported in the included studies (Fig. 4). The overall AUCs were 0.70 (0.68–0.72), 0.70 (0.63–0.76), 0.68 (0.64–0.73), and 0.67 (0.63–0.71) for PSI, A-DROP, I-ROAD, and CURB-65 scores, respectively. No significant differences were observed (p = 0.66, I² = 0%).

Comparison of overall AUC for PSI, A-DROP, I-ROAD, and CURB-65.