Abstract
Background
The Japanese severity criteria for acute pancreatitis (AP), which consist of a prognostic factor score and contrast-enhanced computed tomography grade, have been widely used in Japan.
Objective
This large multicenter retrospective study was conducted to validate the predictive value of the prognostic factor score for mortality and complications in severe AP patients in comparison to the Acute Physiology and Chronic Health Evaluation II (APACHE II) score.
Methods
Data of 1159 patients diagnosed with severe AP according to the Japanese severity criteria for AP were retrospectively collected in 44 institutions.
Results
The area under the curve (AUC) for the receiver-operating characteristic curve of the prognostic factor score for predicting mortality was 0.78 (95% confidence interval (CI), 0.74–0.82), whereas the AUC for the APACHE II score was 0.80 (95% CI, 0.76–0.83), respectively. There were no significant differences in the AUC for predicting mortality between two scoring systems. The AUCs of the prognostic factor scores for predicting the need for mechanical ventilation, the development of pancreatic infection, and severe AP according to the revised Atlanta classification were 0.84 (95% CI, 0.81–0.86), 0.73 (95% CI, 0.69–0.77), and 0.83 (95% CI, 0.81–0.86), respectively, which were significantly greater than the AUCs for the APACHE II score; 0.81 (95% CI, 0.78–0.83) for the need for mechanical ventilation (
Conclusion
The prognostic factor score has an equivalent ability for predicting mortality compared with the APACHE II score. Regarding the ability for predicting the development of severe complications during the clinical course of AP, the prognostic factor score may be superior to the APACHE II score.
Introduction
Acute pancreatitis (AP) is an inflammatory disease resulting from the self-digestion of the pancreas due to the activation of pancreatic enzymes. The overall prevalence of AP ranges from 36–78 cases per 100,000 people annually.1–4 The clinical course of AP varies substantially, from a mild disease treated conservatively to a severe disease resulting in multiple organ failure, infected pancreatic necrosis, and mortality.5–7 Recently, the overall mortality for AP has decreased partly due to the widespread use of guidelines for the management of AP.8,9 Nonetheless, severe AP is still associated with high mortality, ranging from 3–24%.10–13 Early identification of patients with severe AP who are at high risk for complications and unfavorable outcomes is likely to further decrease mortality.
The Japanese severity criteria for acute pancreatitis of the Japanese Ministry of Health, Labour and Welfare.
BUN: blood urea nitrogen; CRP: C-reactive protein; CT: computed tomography; LDH: lactate dehydrogenase; PaO2: partial pressure of oxygen in blood; SIRS: systemic inflammatory response syndrome.
Measures in SIRS diagnostic criteria: (a) temperature >38℃ or <36℃; (b) heart rate >90 beats/min; (c) ventilatory rate >20 breaths/min or PaCO2 <32 torr; (d) white blood cell (WBC) count>12,000 cells/mm3 <4000 cells/mm3, or >10% immature (band) forms.
The aim of this large multi-center retrospective study was to validate the efficacy of the prognostic factor score for predicting mortality and severe complications in patients with severe AP by comparing to the Acute Physiology and Chronic Health Evaluation II (APACHE II) score, 18 which is the globally used severity assessment method.
Materials and methods
Study design
This retrospective multi-center cohort study was performed at 44 institutions in Japan. Twenty-five of the institutions were tertiary academic medical centers, and the other institutions were community hospitals with intensive care units. In each participating institution, all consecutive patients classified as severe AP within 72 h after admission according to the Japanese severity criteria for AP between 1 June 2009–31 December 2013 were enrolled.
Patient data, including demographic, clinical, laboratory, and radiological information, were retrospectively collected. All patients were followed up until hospital discharge.
For the purpose of this study, the total prognostic factor score and the APACHE II score were calculated for all enrolled patients at the diagnosis of severe AP. Based on calculated scores and clinical outcomes of all patients, we assessed the ability of the prognostic factor score to predict hospital mortality, the development of severe complications, the need for surgical interventions, and severe AP according to the revised Atlanta classification 19 by comparing with the APACHE II score.
This study was a post-hoc analysis using data from a retrospective multicenter cohort study to evaluate the effectiveness of continuous regional arterial infusion of protease inhibitors for patients with severe AP. 20 This study was registered at the University Hospital Medical Information Network Clinical Trials Registry (registry number: 000012220), and approved by the Institutional Review Board or the Medical Ethics Committee at each institution. The study was conducted and reported in accordance with the Strengthening Reporting of Observational Studies in Epidemiology guidelines. 21
Definitions
AP was diagnosed if the patient presented with at least two of the following three features: (a) acute abdominal pain and tenderness in the upper abdomen; (b) elevated levels of pancreatic enzymes in the blood and urine; or (c) abdominal findings of acute pancreatitis as detected by ultrasonography, CT, or magnetic resonance imaging. 14 Severe AP, based on the Japanese severity criteria, was diagnosed when the total prognostic factor score was ≥3 or the contrast-enhanced CT grade was ≥2 (Table 1). When patients were diagnosed with mild pancreatitis at admission, we repeatedly evaluated the severity during the early stages of AP under the Japanese guideline. 14
Severe complications during hospitalization included pulmonary failure, renal failure, and pancreatic infection. Pulmonary failure and renal failure were defined as a need for mechanical ventilation and for hemodialysis, respectively. Pancreatic infection was defined as the presence of bacteria based on blood culture or local culture obtained by percutaneous, image-guided, or endoscopic fine-needle aspiration, or the presence of extra-luminal gas in the pancreatic and/or peripancreatic tissues on contrast-enhanced CT. 19 Surgical interventions included percutaneous, endoscopic, laparoscopic, or laparotomy drainage, necrosectomy for infected acute necrotic collection, or walled-off necrosis, interventional radiology, or endoscopic treatment for bleeding.
Statistical analysis
The relationships between the prognostic factor score and hospital mortality were determined by univariate logistic regression analysis. To evaluate the predictive accuracy of scorings systems for each outcome, receiver-operating characteristic (ROC) curves were generated, and the area under the curve (AUC) with 95% confidence interval (CI) was calculated. The Japanese prognostic factor score was compared with the APACHE II score by Delong's test.
22
A
Results
Patient characteristics
Characteristics of severe acute pancreatitis patients analyzed in this study.
APACHE: acute physiology and chronic health evaluation; CECT: contrast enhanced CT; ERCP: endoscopic retrograde cholangiopancreatography; SD: standard deviation.
Regarding the primary outcome, 150 patients (12.9%) died during hospitalization. During the clinical course of AP, 358 patients (30.9%) required mechanical ventilation due to pulmonary failure. Hemodialysis for acute renal failure was performed in 174 patients (15.0%) without pre-existing renal disease. Pancreatic infection occurred in 143 patients (12.3%), and surgical interventions were required in 179 patients (15.4%).
Relationship between the prognostic factor score and the revised Atlanta classification
Relationship between prognostic factor score and revised Atlanta classification.
AP: acute pancreatitis.
Mortality according to the prognostic factor score
The mortality according to the prognostic factor score is shown in Figure 1. The mortality for each prognostic factor score was as follows: 1.3% (2/153) for score = 0, 4% (7/176) for score = 1, 7.1% (14/197) for score = 2, 7.9% (14/177) for score = 3, 13.3% (20/151) for score = 4, 15% (16/107) for score = 5, 34% (32/94) for score = 6, 37.7% (23/61) for score = 7, 44.1% (15/34) for score = 8, and 77.8% (7/9) for score = 9. A significant relationship was observed between the prognostic factor score and the hospital mortality rate by univariate logistic regression analysis ( Mortality rate according to prognostic factor score.
The ability of the prognostic factor score, APACHE II score, and combination score (prognostic factor score plus APACHE II score) to predict clinical outcomes
Figure 2 and Table 4 show ROC curves and AUCs with 95% CI, respectively. For predicting hospital mortality, the AUCs of the prognostic factor score were not significantly different from AUCs of APACHE II score (0.78 vs 0.80, Receiver operator characteristic curves of different severity assessment methods for prediction of severe outcomes. Area under the receiver operating characteristic curves of prognostic factor score and Acute Physiology and Chronic Health Evaluation (APACHE) II score for prediction of severe outcome. SAP: severe acute pancreatitis. The 95% confidence interval is shown in parentheses.
The AUC of the combination score (the prognostic factor score plus APACHE II score) was significantly higher than that of either the prognostic factor score or APACHE II score for predicting hospital mortality (0.81 vs 0.78,
In subgroup analysis investigating etiological differences, the AUCs of the prognostic factor score for predicting hospital mortality were 0.76 in the alcohol group, 0.79 in the cholelithiasis group, 0.78 in the idiopathic group, and 0.78 in the other etiologies group.
Sensitivity, specificity, positive likelihood ratio, and negatives likelihood ratio of the prognostic factor score and APACHE II score
Sensitivity, specificity, positive predictive value, positive likelihood rate, and negative likelihood rate of the prognostic factor score and Acute Physiology and Chronic Health Evaluation (APACHE) II score for prediction of hospital mortality using best cut-off values.
LR: likelihood rate.
The 95% confidence interval is shown in parentheses.
Discussion
In this large multicenter study, the ability to predict hospital mortality and severe complications using the prognostic factor score in the Japanese severity criteria for AP was comparable to the APACHE II score. The prognostic factor score has the advantage of assessment simplicity compared with the APACHE II score, and therefore, appears to be a more practical triage tool during the early stages of AP.
Treatments for severe AP, including close monitoring of vital signs, fasting, fluid resuscitation, and analgesic administration should be initiated immediately after establishing the diagnosis. It is also extremely important to correctly assess AP severity to determine the need for early transfer to specialty care centers,7,8,14,23 because intensive care based on rigorous physiological monitoring and specific therapeutic procedures are associated with improved survival of severe AP patients. For this reason, the Japanese severity criteria were developed to manage patients with AP during the early stage of admission and used for clinical evaluations and public financial support in Japan. In the current study, we evaluated the accuracy of the prognostic factor score in the Japanese severity criteria for predicting disease prognosis.
The prognostic factor score consists of nine clinical and biochemical parameters that can be easily obtained in most local hospitals. In this scoring system, each parameter is assessed using a binary score (positive: 1; negative: 0), and the sum of all positive parameters is represented as the total score using a scale of 0–9. 14
In the current study, we demonstrated that a higher prognostic factor score was strongly correlated with higher mortality. These results provide strong evidence that the prognostic factor score can be used to stratify AP patients accurately according to their risk of mortality. To further evaluate the ability of the prognostic factor score to predict mortality, the AUC for the prognostic factor score was compared with that for the APACHE II score. The prognostic factor score had a reasonably high AUC of 0.78, which was comparable to the AUCs for the APACHE II score (0.80). In previous studies, the AUCs for the prognostic factor score ranged from 0.798–0.822, and the AUC for the prognostic factor score was higher than the Ranson’s score, Glasgow score, and APACHE II score.16,24 From these results, the prognostic factor score can be considered non-inferior to international prognostic evaluation scores for predicting mortality.
The development of organ failure during the early phase of AP is associated with an increased risk of mortality.19,25,26 In addition, infected pancreatic necrosis during the late phase of AP and undergoing surgical interventions are associated with a significant mortality risk. 27 Therefore, it is also meaningful to evaluate the accuracy of the severity assessment scores for predicting the occurrence of these fatal conditions in addition to mortality. In the present study, organ failure included pulmonary failure and renal insufficiency. The AUCs of the prognostic factor score for predicting the need for mechanical ventilation, renal replacement therapy due to renal failure, pancreatic infection, and surgical interventions were 0.84, 0.83, 0.73, and 0.69, respectively, which were comparable to or partially greater than the AUC for the APACHE II score. In a previous cohort study as well, the prognostic factor score was found to be the best scoring system for the prediction of persistent organ failure at 48 h of hospitalization among existing scoring systems. 15
The APACHE II score, which was originally designed to assess the severity of patients with acute illness admitted to intensive care units in the 1970s, is still the most widely used scoring system for severity assessment. It has been selected as a reference standard in numerous studies to evaluate proposed prognostic scoring systems or to identify individual risk factors for severe outcomes. 18 However, the APACHE II score requires the evaluation of 14 parameters with weighting based on the degree of deviation from reference values using the worst data during initial 24 h after admission. Therefore, the scoring system needed 24 h for the final determination of AP severity and is complex and difficult to use in clinical practice. Furthermore, the scoring system is not specific to AP and includes some parameters that are not directly associated with the severity of AP, such as serum sodium and potassium. In a previous study using a large sample of population-based data, only 2.2% of included patients with AP had complete data for the APACHE II classification. 28 The ideal classification system for identifying patients at high risk of mortality should be easy to calculate, consisting of readily available parameters, and accurate.29,30 According to our results, the ability of the prognostic factor score for predicting mortality was comparable to the APACHE II score despite fewer and simpler evaluation parameters. To the best of our knowledge, this was the first large multicenter study in which the prognostic factor score was compared with another scoring system.15–17
For prediction of mortality, the combination score (prognostic factor score plus APACHE II score) was significantly better than the prognostic factor score alone, while for prediction of other severe outcomes, the combination score was not significantly different from prognostic factor score alone. For practical purposes, the combination of prognostic factor score and APACHE II score is cumbersome to use due to increased number of assessment parameter, and therefore its utility will be limited.
In our study with missing data of contrast-enhanced CT in 41 patients, we avoided a direct comparison of the AUCs of prognostic factor score with the AUCs of contrast-enhanced CT grade. Contrast-enhanced CT is an accurate and effective method to evaluate extra-pancreatic extensions of inflammation and pancreatic necrosis. In particular, the identification of pancreatic necrosis on contrast-enhanced CT during an early stage of AP has been reported to be an accurate predictive indicator for severe outcomes.31,32 In the Japanese guidelines for the management of AP, however, the classification of the contrast-enhanced CT grade is not mandatory for the severity assessment in an early phase of AP, but rather, the severity assessment on the basis of the prognostic factor score is recommended. 33 Therefore, we conducted this retrospective study with focusing on evaluating the predictive value of prognostic factor score.
This study had several limitations. Firstly, our study subject was limited to severe AP diagnosed by the Japanese severity criteria. This was because our primary goal was to determine the ability of the prognostic factor score for predicting hospital mortality, while the mortality rate of mild AP was extremely row. 4 Therefore, it was reasonable to assess the efficacy of the prognostic factor score in the severe AP population. Secondly, although patients were treated according to Japanese guidelines for the management of AP, 14 there may be differences in the treatment strategies of AP and the indication for surgical or endoscopic interventions of pancreatic infection depending on participating institutions, which might have affected patient outcomes. Thirdly, prognosis factor scores were calculated by individual institutions, not by the data center. However, since data collection and severity stratification in the prognosis factors were quite simple as mentioned above, the data quality was considered to be adequate for the analyses. Fourthly, in this retrospective study, it was impossible to evaluate predictive ability for the development of cardiovascular complications because we did not collect clinical data on the assessment of cardiovascular complications according to the modified Marshall scoring system used in the revised Atlanta classification. 34
In conclusion, the prognostic factor score has an equivalent ability for predicting mortality compared with the APACHE II score. Regarding the ability for predicting the development of severe complications during the clinical course of AP, the prognostic factor score may be superior to the APACHE II score. The prognostic factor score is designed to allow clinicians to conduct evaluations based on readily available parameters and, therefore, appears to be a more practical tool than the APACHE II score.
