Development and validation of a nomogram to predict bacterial blood stream infection

Abstract

Objective

To identify the risk factors of bacterial blood stream infection (BSI) and construct a nomogram to predict the occurrence of bacterial BSI.

Methods

Blood stream infection is characterized by a systemic infection patient with positive blood culture and has one or more clinical symptoms, such as fever (body temperature > 38 °C) or hypothermia (body temperature < 36 °C), chills, hypotension, oliguria, or high lactic acid levels. The study dataset was randomly divided into a 70% training set and a 30% validation set. Univariate logistic analysis, least absolute shrinkage and selection operator (LASSO) regression analysis, and random forest algorithms were utilized to identify the potential risk factors for BSI. Independent risk factors identified by multivariate logistic analysis were used to construct a nomogram. The discriminative ability, calibrating ability, and clinical practicality of the nomogram were evaluated using the receiver operating characteristic curve, calibration curve, and decision curve analysis.

Results

A total of 195 bacterial BSI patients were enrolled. gender, Acute Physiology and Chronic Health Evaluation-II (APACHEII) score, nCD64 index, erythrocyte sedimentation rate (ESR), procalcitonin (PCT), C-reactive protein (CRP), Interleukin-6 (IL-6), lymphocyte count, T-cell count, B-cell count, NK-cell count, Interleukin-8 (IL-8), Interleukin-10 (IL-10) and Interleukin-17A(IL-17A) were independent risk factors for BSI. The nomogram model exhibited excellent discrimination with an area under the curve (AUC) of 0.836 (95% CI 0.653–0.874) in the training set and 0.871 (95% CI 0.793–0.861) in the validation set. The calibration curve indicated satisfactory calibration ability of the predictive model. Decision curve analysis revealed that the nomogram model had good clinical utility in predicting bacterial BSI.

Conclusion

Overall, this study successfully identified five risk factors for BSI patients and developed a nomogram, offering individualized diagnosis and risk assessment to predict bacterial BSI in infected patients.

Introduction

Blood stream infection is one of the most serious systemic infections and a leading cause of mortality worldwide [1], with studies indicating that BSI-related mortality rates can be as high as 35.9% within 30 to 90 days post-infection [2], and the mortality rate reaches 50% in Candida BSIs [3]. Delay of appropriate antibiotic therapy in the setting of septic shock may result in a 7.6% increase in hourly mortality [4]. Therefore, early and accurate diagnosis of BSI would be essential to improve clinical outcomes in BSI.

Blood culture remains the"gold standard"for the diagnosis of BSI, allowing the identification of a wide range of pathogens in the blood [5]. However, even if a clinical diagnosis of BSI is considered due to the symptoms, there are many reasons that affect the diagnostic efficiency. Firstly, time-consuming blood culture requires at least 24–48 h to isolate and identify the bacteria in positive samples. Secondly, low pathogen loads and slow growth of pathogens make it difficult to incubate and subsequently identify [6]. Thirdly, contamination and incubation time of blood culture usually lead to unnecessary antimicrobial therapy, prolonged hospitalization and increased healthcare costs [7]. Therefore, identifying risk factors for BSI would be clinically important. At present, biomarkers used to diagnose BSI are mainly blood routine test (BRE), PCT, and CRP. There are variabilities among different studies that explored the risk factors of BSI [8, 9]. This study aims to identify the potential risk factors of BSI and construct a nomogram that can be used to personalize clinical decision-making and improve patient care of BSI.

Nomogram is a simple picture tool for predicting clinical outcomes. Through a literature search, we found that few studies have developed nomograms predicting the risk of developing BSI in hospitalized patients. In this study, we collected relatively comprehensive detailed indicators of demographic, immune inflammation and other associations, aiming to investigate the important predictors associated with BSI. In addition, we developed a nomogram to predict the risk of BSI. Finally, we have conducted internal validation of the predictive model, the prediction performance of the model was satisfactory. Such predictive models can help clinicians make correct clinical decisions at earlier time points, thereby avoiding delayed diagnosis and treatment of infection.

Study population

We enrolled a total of 204 patients with positive conventional microbiologic cultures admitted at The First Affiliated Hospital of Nanchang University, China, between April 2023 and September 2023. The blood stream infection group refers to blood bacterial infected individuals who have been cultured positive for peripheral blood samples collected twice or multiple times from different parts of the body, and the focal infection group showed negative blood culture but positive bacterial culture in other specimens. The exclusion criteria included (1) malignant disease; (2) autoimmune system diseases; (3) long-term immunosuppressive treatment; (4)patients intake antibacterial agent; (5) incomplete clinical data; (6) unknown final diagnosis. The patients peripheral blood samples were drawn for the laboratory test on the first day of diagnosis to be BSI. In the meantime, we recorded the demographic and clinical information, and the physiological indicators (body temperature, heart rate, breathing rate, APACHE-II score) and routine laboratory results were collected from the electronic medical records. All the participants provided signed informed consent before participating in the study. According to the diagnostic criteria, a total of 9 patients were excluded for above reasons. The final 195 enrolled patients were randomly divided into a training set (70%, n = 136) and a validation set (30%, n = 59); the distribution of patients is presented in the study flowchart (Fig. 1).

Fig. 1

Flowchart of patient selection

Data collection

The data were obtained from the medical records of our Lab center. Clinical symptoms, demographic characteristics, comorbidities, and laboratory data of patients were collected at the time of admission. Clinical symptoms include fever, abdominal pain, difficulty urinating, skin breakage, cough, jaundice, headache. Demographic characteristics include age, sex, APACHEII score. Comorbidities include acute coronary syndrome, diabetes mellitus, chronic obstructive pulmonary disease and chronic kidney disease. Laboratory data included nCD64 index, ESR, CRP, PCT, Treg cells (%), CD8 + CD28 + T (%), lymphocyte count and lymphocyte subtype percent and various cytokines.

Statistical analysis

In this study, all missing data were filled by multiple interpolation methods using the ‘mice’ package in R software, which is a well-established method for dealing with missing values. Laboratory parameters had < 11% missingness, while clinical variables showed < 5% missing data. Continuous variables that follow a normal distribution were represented as mean ± standard deviation and compared using the t-test. Continuous variables that do not follow a normal distribution were represented as median (interquartile range) and compared using the Mann–Whitney U test. Categorical variables were presented as frequencies and percentages (%) and compared using the Chi-square test. The study dataset was randomly divided into a training set for developing the nomogram model and a validation set for validating it. The table for patient characteristics was generated by ‘CBCgrps’ package. Univariate logistic analysis, least absolute shrinkage and selection operator (LASSO) regression analysis were employed to determine the potential risk factors in the training set. For univariate logistic analysis, variables with p < 0.05 were regarded as potential biomarkers. LASSO regression analysis was chosen for its ability to penalize the variables to reduce the risk of overfitting. For LASSO regression, the optimal tuning parameter λ with tenfold cross-validation was 0.078, following the one standard error of the minimum criteria. At this λ optimal value (absolute value of coefficients > 0.001), the 6 variables were identified as potential biomarkers. The variables obtained by the above 2 methods were included and the backward stepwise regression was used in multivariate logistic analysis to determine the final model. Variance inflation factor (VIF) ≤ 5 signified the absence of collinearity between the final model variables. Based on the independent variables, the nomogram for predicting BSI was developed by the regplot package in R software. The discriminative ability of the nomogram was evaluated using the receiver operating characteristic (ROC) curve in both the training and validation sets. Moreover, decision curve analysis (DCA) was applied to assess the clinical practicality of the nomogram. All statistical analyses were performed using R software (version 4.3.1). p < 0.05 was considered statistically significant.

Results

Patient characteristics

In total, 195 patients with bacterial infections were recruited from The First Affiliated Hospital of Nanchang University in our study (Fig. 1). The cohort was divided into two groups of focal infections and blood stream infection, and separately divided into a training set and a validation set. The results of statistical analysis between the training and validation sets are presented in Table 1.

Table 1 Characteristics of the training and validation sets

Compared with the validation set, patients in the training set had a higher IL-12p70 (p < 0.05). The characteristics of blood stream infection group and focal infection group in the training set were compared in Table 2.

Table 2 Characteristics of BSI and focal infection patients in the training set

Patients with blood stream infection exhibited higher inflammation indicators levels include CRP, nCD64 index, ESR, PCT, CRP, IL-6, IFN-γ,IL-8, IL-10 and IL-17 A. Additionally, they had lower levels of lymphocyte count, T-cell count, B-cell count and NK-cell count compared to those with focal infections (p < 0.05). Furthermore, a significant difference was observed in the indicators reflecting severity of illness between the 2 groups (p < 0.05).

Screening for potential risk factors

Univariate logistic regression analysis showed that gender, APACHE II score, nCD64 index, ESR, PCT, CRP, IL-6, lymphocyte count, T-cell count, B-cell count, NK-cell count, IL-8, IL-10 and IL-17 A were the potential risk factors of blood stream infection (Table 3).

Table 3 Univariate logistic regression analysis for blood stream infection patients

For LASSO regression, the following predictors were identified as risk factors for refractory peritonitis: gender, IL-4, PCT, APACHEII score, nCD64 index and IL-17 A (Fig. 2).

Fig. 2

LASSO regression analysis for BSI. Selection of tuning parameter (λ) using tenfold cross-validation (A) and LASSO coefficient profiles of variables (B)

Potential biomarkers obtained by the above methods were included in multivariate logistic regression analysis and backward stepwise regression was applied to identify the most reliable predictors. The results showed that gender, APACHE II score, nCD64 index, IL-6 and IL-4 were significant independent risk factors for blood stream infection (Table 4).

Table 4 Multivariate logistic regression analysis of BSI in the training set

Nomogram establishment

The prediction model was established based on the training set. Apply the predictors obtained from multiple logistic regression analysis to construct a nomogram, and scores were assigned to the selected variables based on their regression coefficients (Fig. 3).

Fig. 3

Nomogram for predicting BSI. *means p < 0.05; **means p < 0.01; ***means p < 0.001

The cutoff value was 0.451, with a sensitivity of 0.653, and specificity of 0.874 based on the Youden index.

Discrimination and calibration

ROC curve was plotted to examine the discriminative ability of the nomogram (Fig. 4).

Fig. 4

The ROC curves of nomogram for predicting BSI in the training set (A) and validation set (B)

The AUC values of the nomogram for predicting blood stream infection were 0.836 (95% CI 0.716—0.847) in the training set and 0.871 (95% CI 0.627—0.855) in the validation set, indicating its excellent ability to discriminate patients with blood stream infection. The calibration curves for the nomogram were plotted and consistent between the observation and prediction (Fig. 5), indicating that the predictive model has a good calibration capability.

Fig. 5

The calibration curves of nomogram for predicting BSI in the training set (A) and validation set (B)

Clinical practicality

To evaluate the clinical practicality of the nomogram, the decision curves were plotted in both the training and validation sets (Fig. 6). The results showed that the nomogram model exhibits good clinical practicality in predicting blood stream infection.

Fig. 6

The decision curves of nomogram for predicting BSI in the training set (A) and validation set (B)

Discussion

Blood stream infection is the general term for sepsis and bacteremia, there is a close association with other infections such as pneumonia and urinary tract infections. The increase in antimicrobial resistance (AMR) and its direct link to sepsis have made blood stream infection a major concern in the public health field [10]. The time-consuming blood culture impelled the development of many testing strategies to reduce hours, including direct detection from blood samples by PCR and metagenomic sequencing of cell-free DNA in plasma [12]. In addition, emerging testing technologies have become prevalent in recent years, and methods such as proteomics, gene expression analysis, and gene expression profiling have begun to be used in the diagnosis of infectious diseases [11,12,13]. Unfortunately, these detection methods have some drawbacks such as high-cost and long transit time, therefore restricting its clinical application. There are some predictive inflammatory markers like PCT and CRP that are convenient for continuous monitoring. However, high false-positive and false-negative rates hinder the accuracy of detection. Thus, it is necessary to establish a new type of forecasting tool.

In the present study, a nomogram for predicting bacterial BSI among infected patients was built. The predictive model was primarily based on clinically common variables, including APACHEII, nCD64 index, gender, age, ESR, various lymphocyte counts and cytokines. The nomogram contains 5 variables, including IL-4, IL-6, nCD64 index and APACHEII score of patients. The nomogram has been internally validated and shows good discrimination and clinical utility, yet no external validation was performed due to geographic, ethnic, and drug differences.

The activation of inflammatory mediators (e.g., IL-6, PCT, CRP, nCD64 index) is mainly due to the involvement of the innate immune response. When necrotic tissue and microbes are present, they release detrimental substances into the body. These substances include damage-associated molecular patterns (DAMPs) and pathogen-associated molecular patterns (PAMPs) [14] The release of these detrimental substances activates pattern recognition receptors (PRRs) expressed in innate immune cells, such as toll-like receptors (TLRs) [15]. The activation of these receptors further stimulates rapid proliferation of innate immune cells like macrophages, dendritic cells, and neutrophils, accompanied by the release of numerous cytokines, such as IL-1β, IL-2, IL-6, TNF-α, IL-10, PCT, CRP and nCD64 index[16]. Previous studies have shown that they have a large number of clinical applications in the diagnosis of BSI.

Neutrophil CD64 expression is regulated in a graded fashion that seemingly parallels the degree of inflammatory response to the significant clinical process of infection or tissue injury. Previous studies of CD64 usually focused on its use in sepsis and indicated that CD64 index was highly correlated with the presence of infection [17]. A systematic review of eight studies comprising 1986 patients by Wang et al. regarding the diagnostic value of the nCD64 index in adult patients infection found that the pooled sensitivity and specificity were 76% (95%CI: 73–78%) and 85% (95%CI: 82–87%), respectively, it showed that measuring nCD64 expression values is beneficial for early diagnosis of sepsis in critically ill patients [18]. Similarly, the results of our study showed that CD64 infection index in the blood stream infection group was higher compared with the focal infection group and was a risk factor for BSI.

IL-6 and IL-4 play key roles in inflammatory responses and immune regulation, which may influence the occurrence of BSI. As we know, IL-6 acts on vascular endothelial cells and plays an important role in pathogenesis under pathological conditions, leading to pathological damage and contributing to disease onset and progression [19]. IL-4 is a Th2 cytokine that plays a major role in anti-inflammatory responses and immune regulation and may affect the host's ability to clear infection [20]. However, previous studies have shown that excessive levels of IL-4 and IL-6 trigger an overactive immune response, leading to inflammation damage and multi-organ dysfunction [21, 22]. Accordingly, our study suggests that IL-4 and IL-6 are risk factors for BSI. Moreover, immunocyte including lymphocyte count, T-cell count, B-cell count, NK-cell count were decreased in focal infection group, there were not fully consistent results in previous studies and it may need to be verified by expanding the sample size.

Many reports indicated that APACHE II score was an independent risk factor for death in patients with bacterial BSI[23, 24]. However, the accuracy of the score system and utility in clinical practice may be limited because of its complexity. Our research observed that the APACHE II score was higher in the BSI group compared with the focal infection group and serve as risk factors in patients with BSI; it might be because of the severe symptoms and worse clinical outcomes in blood stream infection group.

Interestingly, gender also showed significant differences in BSI group and male patients are more prevalent. This may be related to the differences of the composition ratio in BSI individuals and immunity in different sexes. It was reported that 80% of patients with CRE infection were males, and male was an independent risk factor for CRE infection in a tertiary teaching hospital of China [25], which was validated in our conclusion. In addition, there were reports indicating that male patients were more likely to receive invasive treatments, such as deep venipuncture, mechanical ventilation, and dialysis in the ICU [26]; these treatments and operations may increase the risk of BSI. In our multivariate analysis, gender remained statistically significant (p = 0.027) even after adjusting for illness severity (APACHE II score), age and other laboratory variables included in the model. Emerging evidence suggested sex-based differences in immune response patterns. We acknowledge that our study was not powered to fully explore sex-specific pathophysiology, and residual confounding by unmeasured factors is possible, the relationship may be more complex than our model could capture, potentially involving interactions between sex hormones. Larger sample sizes may be needed to explore the gender-specific mechanisms in BSI apart from the known differences in immune response involved.

IL-4, IL-6, nCD64 index, measurements can usually be performed by routine venous blood sampling, rapid measurements, and accurate laboratory results. According to the nomogram, clinicians could collect the indicators required and calculate the final score corresponding to the risk of infection in patients with BSI. The nomogram is applied as follows: (I) the clinician measured IL-4, IL-6, nCD64 index and at the patient admission and recorded both gender and APACHEII levels; (ii) the total score was calculated by summing the points determined on the"Point"scale for each predictor variable. Comparison of the"total score"scale and the"Risk"scale yields the risk level of individual BSI; (iii) if the probability is higher than 0.331, patients will be classified as"high risk of BSI infection"with further tests including invasive and/or expensive tests to determine the presence of BSI. Otherwise, they could continue on their usual treatment and follow-up observation.

Limitations of the study

Firstly, as a retrospective study, we obtained a limited number of case observations from the single center, it may lead to a selection bias. Secondly, we could not strengthen the external validation of the model owing to lack of the corresponding data set, it needs to be refined in subsequent work. Thirdly, the calibration curve suggests good agreement between observed and predicted probabilities, but there is no assessment of overfitting, given the current cohort's sample size limitations (n = 195), bootstrapping validation might introduce substantial sampling errors. For future work, we plan to:1. supplement with bootstrapping validation (1,000 replicates) in an expanded cohort (target n = 500); 2. validate the model's generalizability using an appropriate external validation cohort when available. Furthermore, we did not include antibiotic use as a factor in the analysis due to the complexity of antibiotic treatment information; the enrolled patients were not treated with antibiotics prior to the test. Finally, our study focused on a single cutoff value optimized for balanced accuracy, and did not comprehensively evaluate alternative thresholds tailored to different clinical priorities (e.g., maximizing sensitivity vs. specificity). This trade-off analysis should be addressed in future validation studies. Further prospective studies are needed to verify the accuracy of our nomogram before widespread clinical application.

Conclusion

We successfully developed and validated a nomogram based on the LASSO regression and the logistic analysis to predict the occurrence of BSI. We aim to control risk factors, conduct timely targeted active surveillance and identify early BSI patients based on the nomogram score, hence guide further treatment and prognosis of BSI.