Abstract: Correctly identifying when a hemorrhagic patient needs immediate medical attention to prevent acute hypotensive episode (AHE) is vital in the short- and long-term care, but is often complicated due to the physiological responses in the sympathetic and parasympathetic nervous systems that mask symptoms until a significant amount of blood loss has occurred. These physiological responses affect the arterial blood pressure waveform, changing both dynamics and waveform morphology. Through the use of Markov chain analysis of the arterial blood pressure waveform, we first analyzed patient blood pressure waveforms from a challenge dataset published by Computing in Cardiology 2009 and the MIMIC III database. Markov chain analysis was applied to 20-second intervals over the entirety of a patient’s known acute hypotensive episode. Each interval or segment is one second apart from the previous segment with a nineteen second overlap. The mixing rate (2nd largest eigenvalue of the transition probability matrix) was determined for all segments. A subset of patients showed a Pearson correlation coefficient with shock index (SI), i.e., with the ratio of heart rate and systolic blood pressure, similar to a previous swine study. These patients (mean correlation coefficient -0.423 ± 0.32, median -0.352) were found to have been administered pressors (vasoconstrictors), compared to patients who were not administered pressors (mean correlation coefficient 0.392 ± 0.29, median 0.447). Patients were also analyzed based on diagnoses of gastrointestinal bleeding by the ICD- 9 code, and mixing rate results were compared between patients in this subgroup and found to have no significance as a metric of predicting acute hypotensive episodes. Aisthesis Volume 9, 201837 Second Largest Eigenvalue of the Transition Probability Matrix for the Markov Chain Constructed from the Arterial Blood Pressure Waveform is Not Correlated to Shock Index in Hemorrhagic Human Subjects by Brett Collar, Marvi Bikak, Paul Brown, Zachary Pitluk, and Mohammad Adibuzzaman Introduction Hemorrhagic shock is a condition produced by rapid and significant loss of intravascular volume, which may lead sequentially to hemodynamic instability, decreases in oxygen delivery, decreased tissue perfusion, cellular hypoxia, organ damage, and death. The primary goals are to stop the bleeding and to restore circulating blood volume, as hemorrhagic shock can rapidly become fatal. Resuscitation may well depend on the estimated severity of hemorrhage. Hemorrhage results in over 80% of operating room deaths after major trauma and almost 50% of all deaths in the first 24 hours of trauma care [1]. Identifying occurrence of hemorrhage in a timely manner is the cornerstone of medical and surgical management. However, one of the major limitations in accurately identifying a state of hemorrhage is the poor predictive ability of heart rate, mean arterial pressure, and shock index [2]. These are the most commonly used physiological parameters which help guide clinicians to diagnose hemorrhage. In other words, developing a method to identify patients that require immediate medical care is of critical importance [2]. Heart rate variability is one such method with the potential to identify hemodynamic instability caused by hemorrhage [3]. Previous literature suggests that reductions in vagal activity assessed with heart rate variability or baroreflex sequences may represent identifiable early markers of hemorrhage [4]. Hypovolemia triggers a simultaneous reduction of the parasympathetic nervous system and activation of the sympathetic nervous system in an attempt to increase heart rate and compensate the drop in blood pressure. As a result, clinical signs present during early stages of hemorrhage may be ignored due to the mean arterial pressure remaining stable. Only after a significant loss of blood will a change in blood pressure be identified, at which point medical interventions could be limited. Clinically, abnormal shock index values, defined as the ratio of heart rate (HR) to systolic blood pressure (SBP), have been demonstrated to portend worse outcomes in traumatically injured patients [5]. Markov chain Second Largest Eigenvalue of the Transition Probability Matrix Aisthesis Volume 9, 201838 methods may describe changes in hemodynamic instability prior to traditional vital signs, potentially providing an early indicator of hemorrhage. A Markov chain is defined as a system with different states, where the transition probability from one state to the next depends only on the current state, the Markov assumption [6]. Regular Markov chains have a limit distribution or steady state, where the mixing rate of a Markov chain represents how fast the system is approaching the steady state. Eigenvalues of Markov chains display changes in system dynamics that are not captured by other nonlinear methods, such as Poincaré plots [4]. These eigenvalues of Markov chains represent changes in the dynamics of arterial blood pressure (ABP) waveforms as the body attempts to compensate for hypovolemia. An empirical Markov chain can be constructed from ABP recordings. As the system dynamics and waveform morphology change, the system will approach steady state faster or slower and this can be observed through the mixing rate. An animal study on hemorrhagic swine showed strong correlation with shock index and the mixing rate of the arterial blood pressure [7]. We studied the applicability of the method in hemorrhagic human subjects from the intensive care unit using two different data sets. We hypothesized that if a specific subset of patients was found to exhibit strong correlations between mixing rate and traditional biomarkers, these patients would share a demographic detail that explained the behavior. Additionally, a larger patient population collectively sharing similar clinical diagnoses was analyzed to determine whether specific conditions affected clinical findings (mean correlation coefficient with SI -0.0087 ± 0.19, median 0.0080). We also provide the theoretical foundation for the change in the second largest eigenvalue from time series data. Methods A. Previous Work A protocol was approved and performed on a swine model to test the efficacy of the Markov chain mixing rate as a metric for detecting hemorrhage prior to noticeable changes in traditional vital signs [7]. Immature swine (N=7, female, 37.1 ± 15.1 kg (mean ± SD)) were anesthetized and instrumented with bilateral catheters in femoral arteries and veins. Data were collected during a continuous hemorrhage of 10 ml/kg over 30 minutes. Heart rate and beat-by-beat blood pressures (systolic, diastolic, mean) were calculated from the ABP waveform, with shock index (SI) calculated by dividing heart rate by systolic blood pressure. Using a high-frequency (125 Hz) arterial blood pressure waveform, an empirical Markov chain was created from a 20-second period of data by segmenting the range of blood pressures over a fixed number of states, each covering an equal range of pressures. From there, a transition probability matrix was computed as the probability that blood pressure will enter any state given only its current state. The matrix was normalized by dividing each row by the cumulative sum of the row to have a probability distribution. Finally, eigenvalues were determined from the transpose of the transition probability matrix, and the eigenvalue with the second largest magnitude was defined as the mixing rate. The arterial blood pressure waveform was then advanced by one second (125 samples), and the process was repeated to find the next Markov chain mixing rate. These mixing rates were collected in a list and graphed with respect to time to compare with traditional vital signs. Pearson correlation coefficients were determined between the mixing rate and each vital sign (heart rate and systolic blood pressure). The mixing rate and high-frequency ABP waveforms (determined at 125 Hz) were then smoothed using a moving average filter (100 samples window) before computing the correlation coefficients. In an anesthetized pig model, the Markov chain mixing rate of the ABP waveform is strongly correlated with the vital signs (mean correlation coefficient with SI -0.889 ± 0.143, median -0.95) [7]. The relationship between the mixing rate and traditional vital signs suggests that this new marker might be an indicator of impending hemodynamic imbalance. B. Computing in Cardiology Challenge Dataset To evaluate the efficacy of the marker on humans subjects, a dataset published by Computing in Cardiology Challenge (CinC) 2009 and the Medical Information Mart for Intensive Care – III (MIMIC III) database was used [8, 9]. One limitation of the challenge data set was that it does not have Second Largest Eigenvalue of the Transition Probability Matrix Aisthesis Volume 9, 201839 information regarding hemorrhage in the data, and acute hypotensive episode was defined as a 30-minute period in which mean arterial pressure was less than 60 mmHg and greater than 10 mmHg for at least 90% of the 30-minute period. A total of 30 patient blood pressure waveforms were obtained and had mixing rates calculated over a period immediately preceding a known acute hypotensive episode (AHE). As shown in Figure 1, ABP waveforms corresponding to each episode were downloaded over a 70 minute window; the first 10 minutes were used as a noncritical observation window to establish a baseline, and the AHE of interest occurred ~30 minutes into the proceeding 60-minute forecast window. These data were analyzed using a MATLAB script to compute Markov chain mixing rates, and Pearson correlation coefficients were determined between mixing rate and traditional vital signs (HR, ABP). Additionally, the correlation coefficient between mixing rate and shock index was calculated as a metric of the predictive capabilities of mixing rate against clinical triage criteria. Fig. 1. An example of an arterial blood pressure waveform analyzed for Markov chain mixing rates. A 10-min. baseline was used, with the AHE occurring approximately 30 min. into the given 60-min. forecast window. C. GI Bleeding Dataset To find hemorrhagic patients based on clinical diagnosis, and not by AHE (as was the case for the challenge data set), a comprehensive search of the MIMIC III database was performed in order to identify and analyze additional patients suffering from acute hypotensive episodes. Using ICD-9 codes across the entire MIMIC III database, patients were identified based on their respective diagnoses, and data for patients with high-frequency waveform were recorded. For this analysis, the ICD-9 code 578.9 (gastrointestinal bleeding) was chosen, as GI bleeding was considered to be the most similar in behavior to the controlled hemorrhage of the previous swine study. However, while clinical data may exist for individual patients, it is imperative that high-frequency physiological signals, more specifically, ABP, are also present in the waveform database, as only a subset of the subjects from the clinical data have their high frequency waveform data in the matched subset [10]. After searching the entire MIMIC III database, 938 patients were diagnosed with ICD-9 code 578.9, and of those patient records, 162 contained matching high-frequency waveform data. The MIMIC:SciDB platform (v16.9, Paradigm4, Waltham, MA), was created by Paradigm4 in collaboration with Regenstrief Center for Healthcare Engineering (RCHE) and hosted at RCHE. Extensive cleaning of the MIMIC time series data was necessary to ensure the integrity of the data and flag missing data. MIMIC:SciDB stores the 10 TB of time series data in arrays with dimensions [time, intensity, patient ID]. Each cell in the 3-dimensional array contains a value for ECG, HR, ABP, and SpO2, among others. Using SciDB’s streaming API for parallel distributed computation, an algorithm was applied to a subject’s entire hospital stay waveform to identify all episodes of acute hypotension, defined as a 30-minute period during which the mean arterial pressure was below 60 mmHg for at least 90% of the time [11, 12]. An R program was created to locate the corresponding high-frequency data and download the relevant data over the identified window of AHE, including a 30-minute baseline prior to AHE onset. Mixing rate analysis was performed for each episode, and correlation coefficients were calculated between mixing rate and vital signs (HR, ABP), as well as against shock index. Results A. Computing in Cardiology Challenge Dataset Analysis of the corresponding results (Figure 2) showed a small subset of patients displayed correlation coefficients similar to the previous animal study, a result that could possibly be explained by the administration of pressors (vasoconstrictors). Of the selected cohort, patients who were administered pressors accounted for 5 of the 7 largest decreases in Markov chain mixing rate, as shown in Table 1 (see Appendix). Conversely, patients who were Second Largest Eigenvalue of the Transition Probability Matrix Aisthesis Volume 9, 201840 not administered pressors accounted for 4 of the 6 smallest decreases in mixing rate, as shown in Table 2 (see Appendix). The correlation coefficients between the mixing rate and the vital signs of patients who were administered pressors show a resemblance to the results obtained in the previous animal study of swine that were administered pressors and underwent controlled hemorrhage. The mixing rate was inversely correlated with heart rate and shock index, and showed a positive correlation to systolic blood pressure. In the subset of patients who were not administered pressors, virtually none of the previous findings hold true. Patients showed a negative correlation with systolic blood pressure and a positive correlation with shock index, with no significant correlation between mixing rate and heart rate. Fig. 2. High-frequency mixing rate of patients provided by the CinC Challenge dataset. The blue line is the average of all patient data over the course of 60 minutes. Hemorrhage starts at time 0 (red vertical line). B. GI Bleeding Dataset Of the 162 patients identified in the matched subset, 47 were analyzed for heart rate variability. Of those, 8 patients were found to suffer from episodes of acute hypotension, with a total of 81 hypotensive episodes identified within the patient subset. However, after analyzing all hypotensive episodes, correlation values greater than |±0.50| accounted for 4 out of 243 measurements (1.6%) between mixing rate and HR, SBP, and SI. Additionally, correlations values between |±0.25| and |±0.50| accounted for 19 out of 243 measurements (7.8%) between mixing rate and HR, SBP, and SI. Mathematical Justification For the changes in the mixing rate, we hypothesized that the mixing rate change is observed in the Markov chain because of the changes in the density (number of non-zero elements of the matrix divided by the total number of elements) and the self-transition probability (the summation of the probabilities that each state will stay at the same state) of the chain. We provide the mathematical justification of the hypothesis using Gershgorin circle theorem that describes the relationship between eigenvalues and the structural properties of the transition matrix [13]. Gershgorin Circle Theorem Let B be an arbitrary matrix. Then the eigenvalues λ of B are located in the union of the n disks, Where bkk is the diagonal element (self-transition probability) and bkj is the non-diagonal elements for each row (related to density) of the matrix. For the analysis, from equation (1), we get, From (2) and (3), The construction of the Markov chain is such that the summation of each row is 1. As a result, Second Largest Eigenvalue of the Transition Probability Matrix Aisthesis Volume 9, 201841 As more states transition to the other states, the self-transition probability bkk decreases and increases, resulting in a decrease in the lower bound of this inequality. This is what we observed in the empirical analysis after the hemorrhage. We conclude that the lower bound for the second largest eigenvalue decreases with progressive hemorrhage in anesthetic swine, as captured by the algorithm. But the same conclusion did not hold true for hemorrhagic human subjects, as defined by our study. Discussion In the CinC 2009 Challenge dataset, the Markov chain mixing rate of the ABP waveform did not match the results of the previous animal study. However, the mixing rate has a higher correlation coefficient with the shock index for the patients who were administered pressors, suggesting patients administered pressors benefit more from Markov chain analysis than those not administered pressors. For patients administered pressors, the mixing rate had an average correlation of 0.43 with arterial blood pressure, -0.194 with heart rate, and -0.423 with shock index (5 patients analyzed). Among patients not administered pressors (4 patients analyzed), the mean correlation coefficient was -0.386 ABP, -0.075 with heart rate, and 0.392 with shock index. These correlations do not reflect the findings of the previous study. There are limitations for the data sets that we used. For the challenge data set, the patients were assumed to have an acute hypotensive episode as a consequence of hemorrhage, which is not always true. Oftentimes, diagnoses of coronary artery disease, congestive heart disease, or similar complications can lead to symptoms of hypotensive episodes. For the GI bleeding data set, the patients were identified using ICD-9 code for GI bleeding, and an algorithm was applied to define the time for acute hypotensive episode from arterial blood pressure data. It is possible that a patient experiencing an acute hypotensive episode was not having hemorrhage, but rather had another clinical diagnosis, such as septic shock, hypotension resulting from adrenal insufficiency, or as a result of abdominal compartment syndrome. All of the aforementioned conditions can be expected in a patient with a GI bleeding diagnosis. The relationship between higher correlation coefficients for patients with the administration of pressors might be a useful factor to consider for future Markov chain analyses. Hypovolemia ensues when intravascular plasma volume is considerably low. This results in hypotension, which is clinically managed by volume administration (e.g., saline products or blood products). However, if this does not resolve hypotension, pressors are administered, increasing the systemic vascular resistance and hence the blood pressure. Oftentimes, medications administered for anesthesia can result in refractory hypotension requiring pressors, meaning that the patients who were administered pressors may have been experiencing similar symptoms to the previous animal study, leading to the comparable decreases in mixing rate and correlation coefficients between studies. For the data analyzed of patients diagnosed with ICD-9 codes for gastrointestinal bleeding, correlation coefficients greater than ±0.50 accounted for 1.6% of all vital sign measurements. Correlation coefficients between ±0.25 and ±0.50 accounted for 7.8% of all vital sign measurements. From the data currently analyzed within this patient subset, the results do not provide enough evidence to support the claim that mixing rate is a predictive metric for hypotensive events. References [1] D. S. Kauvar, R. Lefering, and C. E. Wade, “Impact of hemorrhage on trauma outcome: an overview of epidemiology, clinical presentations, and therapeutic considerations,” J Trauma, vol. 60, no. 6 Suppl, pp. S3-11, Jun 2006. [2] M. Y. Rady, E. P. Rivers, and R. M. Nowak, “Resuscitation of the critically ill in the ED: responses of blood pressure, heart rate, shock index, central venous oxygen saturation, and lactate,” Am J Emerg Med, vol. 14, no. 2, pp. 218- 25, Mar 1996. [3] J. F. Thayer, S. S. Yamamoto, and J. F. Brosschot, “The relationship of autonomic imbalance, heart rate variability and cardiovascular disease risk factors,” International J of Cardiology, vol. 141, no. 2, pp. 122-131, 2010. Second Largest Eigenvalue of the Transition Probability Matrix Aisthesis Volume 9, 201842 [4] S. Merrill, “Markov chains for identifying nonlinear dynamics,” Nonlinear Dynamical Systems Analysis for the Behavioral Sciences Using Real Data, pp. 401-423, 2010. [5] C. M. Cannon, C. C. Braxton, M. Kling-Smith, J. D. Mahnken, E. Carlton, and M. Moncure, “Utility of the shock index in predicting mortality in traumatically injured patients,” J Trauma, vol. 67, no. 6, pp. 1426-30, Dec 2009. [6] S. Merrill and J. Cochran, “Markov chain methods in the analysis of heart rate variability,” Nonlinear Dynamics and Time Series, vol. 11, p. 11, 1997. [7] M. Adibuzzaman, G. C. Kramer, L. Galeotti, S. J. Merrill, D. G. Strauss, and C. G. Scully, “The mixing rate of the arterial blood pressure waveform Markov chain is correlated with shock index during hemorrhage in anesthetized swine,” (in eng), Conf Proc IEEE Eng Med Biol Soc, vol. 2014, pp. 3268-71, 2014. [8] “Predicting Acute Hyptotensive Episodes: A Challenge from PhysioNet and Computers in Cardiology 2009” (2009). PhysioNet [online]. Available: https://www.physionet.org/ challenge/2009/ [9] A. Johnson et al., “MIMIC-III, a freely accessible critical care database,” Scientific Data, vol. 3, 2016. [10] M. Adibuzzaman, K. Musselman, A. Johnson, P. Brown, Z. Pitluk, and A. Grama, “Closing the data loop: An integrated open access analysis platform for the MIMIC database,” in 2016 Computing in Cardiology Conference (CinC), 2016, pp. 137-140. [11] J. Lee and R. G. Mark, “An investigation of patterns in hemodynamic data indicative of impending hypotension in intensive care,” Biomed Eng Online, vol. 9, p. 62, Oct 25 2010. [12] P. Brown and M. Adibuzzaman, ed, 2018. [13] W. D. James, Applied Numerical Linear Algebra. Philadelphia, USA: SIAM, 1997. Second Largest Eigenvalue of the Transition Probability Matrix Aisthesis Volume 9, 2018 Table 1. Correlation coefficients between mixing rate and vital signs during hemorrhage in patients who were administered pressors. Table 2. Correlation coefficients between mixing rate and vital signs during hemorrhage in patients who were not administered pressors. Appendix 43